Editorial. Reproducible Research in Signal Processing

Reproducible research results become more and more an important issue as systems under investigation are growing permanently in complexity, and it becomes thus almost impossible to judge the accuracy of research results merely on the bare paper presentation. The precise definition of reproducibility varies across disciplines, but it is often closely related to the definitions of repeatability and replicability. In a recent encyclopedia of philosophy of science, reproducibility is described as the repeatability of the process of establishing a factor of the conditions under which the same fact can be observed. In natural science, reproducibility is often related closely to the repeatability of experimental conditions and results. EURASIP became more and more interested in this issue in particular as some of the EURASIP journals would like to make reproducibility an important factor for accepting a submitted paper and by this increase the quality of the papers and in consequence of the entire journal. First trials were not very successful and the idea was born to offer a special issue not focused on any DSP subject but just on the concept of reproducibility. By this it would be possible to recognize which fields in DSP are better suited. We therefore devoted this special issue of the EURASIP Journal of Advances in Signal Processing particularly to research results in signal processing that can be proven to be reproducible. In particular, we expected that the code and the companion data in the submitted paper will be available by the authors at submission time to be checked as part of the review process. If the paper is accepted for publication, then the software must be provided for downloads. Though source code is preferred, the authors must prepare a working object code. In order to facilitate reproducible results, we expected that all codes are provided to reproduce the various tables and figures in the paper. Furthermore, we expect that the code is well documented and a permanent email address as point of contact shall be provided possibly also with a permanent webpage so that fellow researchers can refer to someone if they have questions or simply report bugs they found. It would also be good to see additional testing material that was used to evaluate the correctness of the code. Despite our high expectations at the end only 4 papers were accepted the reason being that among the 31 submissions most papers were immediately rejected as they did not show any sign of reproducibility at all. The finally accepted papers were In the paper entitled "A Full Bi-tensor Neural Tractography Algorithm Using the Unscented Kalman Filter," coauthored by Stefan Lienhard, James Malcolm, Carl Frederik Westin, and Yogesh Rathi, a technique for visualizing neural pathways in human brains by extending an existing framework which uses overlapping Gaussian tensors to model that the signal is described. At each point on the fiber an unscented Kalman filter is used to find the most consistent direction as a mixture of previous estimates and of the local model. Experiments on synthetic data show a reduction of the angular error at fiber crossings and branchings. Tests on in vivo data demonstrate the ability to trace fibers in areas containing crossings or branchings, and the tests also confirm the superiority of using a full tensor representation over the simplified model. In "Internet-based Hardware/Software Co-design Framework for Embedded 3D Graphics Applications," by Chi-Tsai Yeh, Chun-Hao Wang, Ing J. Huang, and Weng-Fai Wong, a hardware/software co-design environment for 3D graphics application development is proposed that includes the 3D graphics software, OpenGL ES application programming interface (API), device driver, and 3D graphics hardware simulators. A 3D graphics system-on-a-chip (SoC) accelerator was developed using transaction-level modeling (TLM). This gives software designers early access to the hardware even before it is ready. On the other hand, hardware designers also stand to gain from the more complex test benches made available in the software for verification. A unique aspect of the framework is that it allows hardware and software designers from geographically dispersed areas to cooperate and work on the same framework. Designs can be entered and executed from anywhere in the world without full access to the entire framework, which may include proprietary components. This results in controlled and secure transparency and reproducibility, granting leveled access to users of various roles. In "The Vienna LTE Simulators--Enabling Reproducibility in Wireless Communications Research," by Christian Mehlführer, Josep Colom Ikuno, Michal Šimko, Stefan Schwarz, Martin Wrulich, and Markus Rupp, a MATLAB-based link and system level simulation environment for UMTS Long Term Evolution (LTE) is proposed. The source codes of both simulators are available under an academic non-commercial use license, allowing researchers full access to standard-compliant simulation environments. Owing to the open source availability, the simulators enable reproducible research in wireless communications and comparison of novel algorithms. The link and system level simulations are connected and the link level simulator serves as a reference to design the system level simulator. The accuracy of the PHY modeling at system level is validated by means of simulations, performed both with bit-accurate link level simulations and PHY-model-based system level simulations. In "Reproducible LTE Uplink Performance Analysis using Precomputed Interference Signals," by Volker Pauli, Muhammad Danish Nisar, and Eiko Seidel, the use of realistic precomputed interference patterns is proposed for LTE uplink performance analysis and testing. The interference patterns are generated via an LTE system-level simulator for a given set of scenario parameters, such as cell configuration, user configurations, and traffic models. The generated interference patterns (some of which are made publicly available) can be employed to benchmark the performance of any LTE uplink system in both lab simulations as well as field trials for practical deployments. It is worth mentioning that the proposed approach can also be extended to other cellular communication systems employing OFDMA-like multiple access with frequency-selective RRM techniques. The proposed approach offers twofold advantages. First, it allows for repeatability and reproducibility of the performance analysis. This is of crucial significance for researchers and developers for analyzing the behavior and performance of their systems, and also for the network operators to compare the performance of competing system vendors. Second, the proposed testing mechanism evades the need for deployment of multiple cells (with multiple active users in each) to achieve realistic field trials, thereby resulting in significant cost (and time) savings in the field trails