Factors systematically associated with errors in subjective estimates of software development effort: The stability of expert judgment

Software metric-based estimation of project development effort is most often performed by expert judgment rather than by using an empirically derived model (although such may be used by the expert to assist their decision). One question that can be asked about these estimates is how stable are they with respect to characteristics of the development process and product? This stability can be assessed in relation to the degree to which the project has advanced over time, the type of module for which the estimate is being made, and the characteristics of that module. In this paper we examine a set of expert-derived estimates for the effort required to develop a collection of modules from a large health-care system. Statistical tests are used to identify relationships between the type (screen or report) and characteristics of modules and the likelihood of the associated development effort being under-estimated, approximately correct, or over-estimated. Distinct relationships are found that suggest that the estimation process being examined was not unbiased to such characteristics.Unpublished[1] A. Albrecht and J. G. Jr. Software function, source lines of code, and development effort prediction: a software science validation. IEEE Transactions on Software Engineering, 9(6):639–648, 1983. [2] B. Boehm. Software Engineering Economics. Prentice-Hall, Englewood Cliffs NJ, USA, 1981. [3] A. Gray and S. MacDonell. Fuzzy logic for software metric models throughout the development life cycle. In Proceedings of the 1999 Annual Meeting of the North American Fuzzy Information Processing Society - NAFIPS, New York NY, USA, 1999. IEEE Computer Society Press. To appear. [4] F. Heemstra and R. Kusters. Function point analysis: evaluation of a software cost estimation model. European Journal of Information Systems, 1(4):229–237, 1991. [5] M. Host and C. Wohlin. A subjective effort estimation experiment. Information and Software Technology, 39:755–762, 1997. [6] R. Hughes. Expert judgement as an estimating method. Information and Software Technology, 38:67–75, 1996. [7] W. Humphrey. A Discipline for Software Engineering. Addison Wesley, 1995. [8] A. Lederer and J. Prasad. Nine management guidelines for better cost estimating. Communications of the ACM, 35(2):51–59, 1992. [9] A. Lederer and J. Prasad. A causal model for software cost estimating error. IEEE Transactions on Software Engineering, 24(2):137–148, 1998. [10] R. Meli. Early function points: a new estimation method for software projects. In Procceedings ESCOM97, Berlin, 1997. [11] T. Mukhopadhyay, S. Vicinanza, and M. Prietula. Examining the feasibility of a case-based reasoning model for software effort estimation. MIS Quarterly, pages 155–171, June 1992. [12] M. Shepperd and C. Schofield. Estimating software project effort using analogies. IEEE Transactions on Software Engineering, 23(12):736–743, 1997. [13] E. Stensrud and I. Myrtveit. Human performance estimating with analogy and regression models: an empirical validation. In Proceedings of the Fifth International Software Metrics Symposium (Metrics’98), pages 205–213, Los Alamitos, California, 1998. IEEE Computer Society Press