A Survey of Systems Engineering Effectiveness: Initial Results

This survey quantifies the relationship between the application of Systems Engineering (SE) best practices to projects and programs, and the performance of those projects and programs. The survey population consisted of projects and programs executed by defense contractors who are members of the Systems Engineering Division of the National Defense Industrial Association. The deployment of SE practices on a project or program was measured through the availability and characteristics of specific SE-related work products. Project Performance was measured through typically available project measures of cost performance, schedule performance, and scope performance. Additional project and program information such as project size, project domain, and other data was also collected to aid in characterizing the respondent's project. Analysis of the survey responses revealed moderately strong statistical relationships between Project Performance and several categorizations of specific of SE best practices. Notably stronger relationships are apparent by combining the effects of more than one the best practices categories. Of course, Systems Engineering Capability alone does not ensure outstanding Project Performance. The survey results show notable differences in the relationship between SE best practices and performance between more challenging as compared to less challenging projects. The statistical relationship between Project Performance and the combination of SE Capability and Project Challenge is quite strong for survey data of this type