Optimal experiments in the presence of a learning effect: a problem suggested by software production

none2In software engineering empirical comparisons of different ways of writing computer code are often made. This leads to the need for planned experimentation and has recently established a new area of application of DoE. This paper is motivated by an experiment on the production of multimedia services on the web, performed at the Telecom Research Centre in Turin, where two different ways of developing code, with or without a framework, were compared. As the experiment progresses, the programmer’s performance improves as he/she undergoes a learning process; this must be taken into account as it may affect the outcome of the trial. In this paper we discuss statistical models and D-optimal plans for such experiments and indicate some heuristics which allow a much speedier search for the optimum. Solutions differ according to whether we assume that the learning process depends or not on the treatments.The paper was written when both authors were members of PRO-ENBIS., a project supported by funding under the European Commission's Fifth Framework 'Growth' Programme via the Thematic Network "Pro-ENBIS" contract reference: G6RT-CT-2001-05059. Its mission was to promote the widespread use of sound science-driven applied statistical methods in European business and industrynoneGIOVAGNOLI A.; ROMANO D.GIOVAGNOLI A.; ROMANO D