Abstract Genetic Improvement is an area of Search Based Software Engineering that aims to apply evolutionary computing operators to the software source code to improve it according to one or more quality metrics. This article describes challenges related to experimental studies using Genetic Improvement in JavaScript (an interpreted and non-typed language). It describes our experience on performing a study with fifteen projects submitted to genetic improvement with the use of a supercomputer. The construction of specific software infrastructure to support such an experimentation environment reveals peculiarities (parallelization problems, management of threads, etc.) that must be carefully considered to avoid future research threats to vali...