uni-oldenburg.de

uni-oldenburg.de Experiments are a corner stone of HCI research. Mobile dis-tribution channels such as Apple’s App Store and Google’s Android Market have created the opportunity to bring ex-periments to the end user. Hardly any experience exists how to conduct such experiments successfully. This article re-ports about five experiments that we conducted by publish-ing Apps in the Android Market. The Apps are freely avail-able and have been installed more than 30,000 times. The outcomes of the experiments range from failure to valuable insights. Based on these outcomes we identified factors that account for the success of experiments using mobile appli-cation stores. When generalizing findings it must be consid-ered that smartphone users are a non-representative sample of the world’s population. Most participants can be obtained by informing users about the study when the App had been started for the first time. Because Apps are often used for a short time only, data should be collected as early as possible. To collect valuable qualitative feedback other channels than user comments and email have to be used. Finally, the inter-pretation of collected data has to consider unpredicted usage patterns to provide valid conclusions