A new data compression method and its application to cosmic shear analysis

Context. Future large scale cosmological surveys will provide huge data sets whose analysis requires efficient data compression. In particular, the calculation of accurate covariances is extremely challenging with an increasing number of observables used in the statistical analysis. Aims. The primary aim of this paper is to introduce a formalism for achieving efficient data compression, based on a local expansion of the observables around a fiducial cosmological model. We specifically apply and test this approach for the case of cosmic shear statistics. In addition, we study how well band powers can be obtained from measuring shear correlation functions over a finite interval of separations. Methods. We demonstrate the performance of our approach, using a Fisher analysis on cosmic shear tomography described in terms of E-/B-mode separating statistics (COSEBIs). Results. We show that our data compression is highly effective in extracting essentially the full cosmological information from a strongly reduced number of observables. Specifically, the number of observables needed decreases by at least one order of magnitude relative to the COSEBIs, which already compress the data substantially compared to the shear two-point correlation functions. The efficiency appears to be affected only slightly if a highly inaccurate covariance is used for defining the compressed data vector, showing the robustness of the method. In addition, we show the strong limitations on the possibility of constructing top-hat filters in Fourier space, for which the real-space analog has a finite support, yielding strong bounds on the accuracy of band power estimates. Conclusions. We conclude that efficient data compression is achievable and that the number of compressed data points depends on the number of model parameters. Furthermore, a band convergence power spectrum inferred from a finite angular range cannot be accurately estimated. The error on an estimated band power is larger for a narrower filter and a narrower angular range, which for relevant cases can be as large as 10% for Δℓ/ℓ ~ 0.1