Data boundary fitting using a generalized least-squares method

This article has been accepted for publication in Monthly notices of the Royal Astronomical Society. © 2009 The Author. Journal compilation © 2009 RAS. Published by Oxford University Press on behalf of the Royal Astronomical Society. Valuable discussions with Guillermo Barro, Juan Carlos Muñoz and Javier Cenarro are gratefully acknowledged. The author is also grateful to the referee, Charles Jenkins, for his useful comments. This work was supported by the Spanish Programa Nacional de Astronomía y Astrofísica under grant AYA2006–15698–C02–02.In many astronomical problems one often needs to determine the upper and/or lower boundary of a given data set. An automatic and objective approach consists in fitting the data using a generalized least-squares method, where the function to be minimized is defined to handle asymmetrically the data at both sides of the boundary. In order to minimize the cost function, a numerical approach, based on the popular DOWNHILL simplex method, is employed. The procedure is valid for any numerically computable function. Simple polynomials provide good boundaries in common situations. For data exhibiting a complex behaviour, the use of adaptive splines gives excellent results. Since the described method is sensitive to extreme data points, the simultaneous introduction of error weighting and the flexibility of allowing some points to fall outside of the fitted frontier, supplies the parameters that help to tune the boundary fitting depending on the nature of the considered problem. Two simple examples are presented, namely the estimation of spectra pseudo-continuum and the segregation of scattered data into ranges. The normalization of the data ranges prior to the fitting computation typically reduces both the numerical errors and the number of iterations required during the iterative minimization procedure.Depto. de Física de la Tierra y AstrofísicaFac. de Ciencias FísicasTRUESpanish Programa Nacional de Astronomía y Astrofísicapu