Channeling the innovation stream: A decision framework for selecting emerging technologies

The proliferation of emerging technologies offers opportunity but also presents challenges to defense acquisition decision makers seeking to incorporate those technologies as part of the acquisition process. Assessment frameworks and methodologies found in the literature typically address the primary focus of a sponsoring organization's interest (e.g., a non-profit 'green' organization's focus on environmental issues). The lack of a holistic, systematic approach for these decisions inhibits the consistent treatment of emerging technology opportunities. The increasing convergence of emerging technologies and the need for multi- or trans-disciplinary collaboration requires a systematic set of considerations to improve consistency of decision making in early stages of the acquisition life cycle. Policy makers and society desire reassurance that defense organizations are incorporating emerging technologies in a safe, understandable, and consistent manner. That desire fuels a need for a systematic approach to consider a full suite of relevant factors when evaluating emerging technology opportunities as part of any management decision making process. This research makes the case for an approach that applies a decision framework for the consideration of emerging technology opportunities during the early stages of defense acquisition in the United States. Results of a cross-sectional survey of U. S. defense acquisition practitioners provide perceptions of the positive value of a systematic decision framework for assessing emerging technology in acquisitions. The survey results also confirm the need for a holistic set of decision factors. In support of the need for a holistic set of decision factors, this research presents a conceptual decision model that offers a systematic approach underpinned by 16 decision factors. The five-layer model is offered as a basis for future research to create an applied decision model for emerging technology opportunities. The results of this research bridge a gap in academic literature and defense acquisition practice with a systematic approach for making decisions about emerging technologies.ABSTRACT Title of Dissertation: CHANNELING THE INNOVATION STREAM: A DECISION FRAMEWORK FOR SELECTING EMERGING TECHNOLOGIES Philip S. Sauer, Doctor of Management, 2010 Dissertation Directed By: Dr. Thomas Mierzwa UMUC Graduate School of Management & Technology The proliferation of emerging technologies offers opportunity but also presents challenges to defense acquisition decision makers seeking to incorporate those technologies as part of the acquisition process. Assessment frameworks and methodologies found in the literature typically address the primary focus of a sponsoring organization’s interest (e.g., a non-profit ‘green’ organization’s focus on environmental issues). The lack of a holistic, systematic approach for these decisions inhibits the consistent treatment of emerging technology opportunities. The increasing convergence of emerging technologies and the need for multi- or trans-disciplinary collaboration requires a systematic set of considerations to improve consistency of decision making in early stages of the acquisition life cycle. Policy makers and society desire reassurance that defense organizations are incorporating emerging technologies in a safe, understandable, and consistent manner. That desire fuels a need for a systematic approach to consider a full suite of relevant factors when evaluating emerging technology opportunities as part of any management decision making process. This research makes the case for an approach that applies a decision framework for the consideration of emerging technology opportunities during the early stages of defense acquisition in the United States. Results of a cross-sectional survey of U. S. defense acquisition practitioners provide perceptions of the positive value of a systematic decision framework for assessing emerging technology in acquisitions. The survey results also confirm the need for a holistic set of decision factors. In support of the need for a holistic set of decision factors, this research presents a conceptual decision model that offers a systematic approach underpinned by 16 decision factors. The five-layer model is offered as a basis for future research to create an applied decision model for emerging technology opportunities. The results of this research bridge a gap in academic literature and defense acquisition practice with a systematic approach for making decisions about emerging technologies. CHANNELING THE INNOVATION STREAM: A DECISION FRAMEWORK FOR SELECTING EMERGING TECHNOLOGIES By Philip S. Sauer Dissertation submitted to the Faculty of the Graduate School of the University of Maryland University College, in partial fulfillment of the requirements for the degree of Doctor of Management 2010 Advisory Committee: Dr. Thomas Mierzwa, Chair Dr. Timothy Reed Dr. Anna Andriasova © Copyright by Philip S. Sauer 2010 ii Dedication I dedicate this dissertation to my family, especially… To my wife Eleni, for 31 years of love and support; To my daughter, Jamie, for blessing us with two beautiful granddaughters over the past five years–Kailah Marie and Angelina Mae; To my daughter, Felicia, for choosing the right path and working so hard to be her best; To my departed father and mother for instilling the importance of hard work and sound values; and, To my Lord, for giving me the knowledge, wisdom, and ability to successfully accomplish this goal. Without your collective support, I would not have achieved this milestone in my life. iii Acknowledgements I owe everlasting gratefulness to the many people who pleasantly involved themselves in helping me undertake and complete this dissertation. While many people have helped me achieve this goal, I specifically acknowledge the following individuals who were especially instrumental in my success. I would like to acknowledge the inspiration provided to a young United States Air Force airman in 1979 by CMSgt (retired) Harold K. “Smitty” Smith. His encouragement to return to education rekindled my interest in academics and led to the completion of a bachelor’s degree in 1984, one of my first steps in this long journey. I also received repeated encouragement from Dr. Myron Hura, a colleague who provided me innovative thoughts, continual encouragement, and sound mentoring since we first met and collaborated in 1995. I appreciate the past 15 years of spirited debate and successful problem solving. At UMUC, I have had the privilege to work with many talented individuals who have made contributions to my research experience. Of note are those members who have participated in the journey since we began in August 2005—Dr. Chris Powell, Dr. Jerry Chadwick, Dr. Ed Zelinski and Dr. Bert Jarreau. I am pleased to call them colleagues and friends. Without their help and support, concluding this journey would not have been possible. I owe special thanks to my dissertation chair, Dr. Thomas Mierzwa, who mentored and facilitated the successful completion of this dissertation. Despite his busy schedule, he always found the time to discuss my research challenges, explore conceptual ideas, and advise me on the mechanics of completing the doctoral program. His support and direction were instrumental in my success, and I am truly grateful for his guidance. iv Table of Contents ABSTRACT ................................................................................................................................... 1 Dedication ...................................................................................................................................... ii Acknowledgements ........................................................................................................................ iii Table of Contents ........................................................................................................................... iv List of Tables ................................................................................................................................ vi List of Figures ............................................................................................................................... vii List of Abbreviations ..................................................................................................................... ix Chapter 1: Introduction and Research Problem .............................................................................. 1 Introduction .................................................................................................................................... 1 Context and Problem Statement...................................................................................................... 4 Purpose ........................................................................................................................................... 5 Thesis ............................................................................................................................................. 7 Research Questions ......................................................................................................................... 7 Scope of Study .............................................................................................................................. 14 Significance of This Study ............................................................................................................ 15 Discussion of Key Terms .............................................................................................................. 17 Dissertation Chapters .................................................................................................................... 23 Chapter 2: Literature Review ........................................................................................................ 25 Introduction .................................................................................................................................. 25 Theoretical Basis of Research ....................................................................................................... 31 Emerging Technology ................................................................................................................... 33 Foresight Analysis—The Identification of Emerging Technology Opportunities ....................... 38 Nano-scale Technology—An Exemplar of the Emerging Technology Challenge ....................... 40 Influence of Emerging Technologies on Innovation Opportunities ............................................. 44 Emerging Technology Decision Making ...................................................................................... 50 DoD’s Capability Based Assessment Approach—A Defense Decision Process Example .......... 54 Systems Considerations in Addressing Emerging Technology Opportunities ............................. 56 Risk – One Aspect of Systems Considerations ............................................................................. 64 Measures – A Second Key Aspect of Systems Considerations .................................................... 68 Existing Technology Assessment and Decision Support Frameworks......................................... 70 Literature Review Conclusion ...................................................................................................... 76 Chapter 3: Conceptual Framework ............................................................................................... 78 Introduction .................................................................................................................................. 78 Description of the Conceptual Model’s Layers ............................................................................ 80 Hypotheses Developed for Study.................................................................................................. 87 Variables for Analysis................................................................................................................... 92 Summary of the Conceptual Model .............................................................................................. 95 Chapter 4: Research Method ......................................................................................................... 96 Research Objective and Process ................................................................................................... 96 v Research Design........................................................................................................................... 97 Data Collection ........................................................................................................................... 100 Survey Development ................................................................................................................... 101 Analysis Approach ...................................................................................................................... 108 Threats to Validity ...................................................................................................................... 109 Research Constraints or Limitations ........................................................................................... 110 Summary of Research Methodology .......................................................................................... 111 Chapter 5: Analysis of Findings and Discussion ........................................................................ 114 Visualization and Presentation of the Data and Analysis ........................................................... 114 Survey Respondent Summary ..................................................................................................... 115 Interpretation of Survey Data...................................................................................................... 118 Pursuing Systematic Insights from the Survey Data................................................................... 119 Perceptions of Management and Decision Processes ................................................................. 136 Decision Factor Perceptions ....................................................................................................... 156 Discussion of Findings ................................................................................................................ 185 The Research Hypotheses are Supported .................................................................................... 187 Answers to the Research Questions ............................................................................................ 194 The Thesis is Supported by the Research Conclusions .............................................................. 197 Chapter 6: Implications for Management ................................................................................... 199 Implications for Management Theory ........................................................................................ 202 Implications for Management Practice ....................................................................................... 203 Recommendations for Future Research ...................................................................................... 206 References .................................................................................................................................. 209 Appendix A – Research Concept Map........................................................................................ 230 Appendix B – Survey Instrument ............................................................................................... 231 vi List of Tables Table 1: Research Questions ........................................................................................................... 9 Table 2: Summary of Propositions Derived from Literature Review ........................................... 76 Table 3: Proposition-Hypothesis-Variable Linkage ..................................................................... 94 Table 4: Summary of Systematic Process Use Perceptions ........................................................ 136 Table 5: Comparison of Three Survey Questions – 7, 8, and 9 .................................................. 144 Table 6: Summary of Decision Process Analysis ....................................................................... 156 Table 7: Decision Factor Importance .......................................................................................... 161 Table 8: Decision Factor Response Layer Perceived Value Summary ...................................... 174 Table 9: Decision Factor Aggregated Perceived Value Summary ............................................. 176 Table 10: Comparison of Aggregated versus Response Layer Factor Rankings........................ 183 Table 11: Research Question Satisfaction .................................................................................. 195 vii List of Figures Figure 1: Research Topics Developed Using a Concept Mapping Approach .............................. 28 Figure 2: Proposed Systematic Framework Model for Emerging Technology Selection Decisions ...................................................................................................................................... 80 Figure 3: Decomposition of the Third Layer of the Conceptual Model ....................................... 85 Figure 4: Context of Methodology Design ................................................................................... 98 Figure 5: Data Collection Model ................................................................................................ 101 Figure 6: Topic Map for Survey Question Grouping and Development .................................... 103 Figure 7: Survey Question Creation Approach ........................................................................... 104 Figure 8: Survey Respondents’ Organizational Affiliation ........................................................ 116 Figure 9: DoD Respondents’ Affiliation..................................................................................... 117 Figure 10: Systematic Process Value to Acquisition Success .................................................... 120 Figure 11: Systematic Process Key to Acquisition Success ....................................................... 121 Figure 12: Systematic Processes Provide Successful Decision Making ..................................... 122 Figure 13: Application of Systematic Approaches by Program Managers ................................. 123 Figure 14: Level of Involvement by Organizational Components ............................................. 125 Figure 15: Use of Structured Criteria by Program Managers ..................................................... 126 Figure 16: Program Manager Use of Processes .......................................................................... 127 Figure 17: Use of Standards ........................................................................................................ 128 Figure 18: Use of Ad Hoc Processes by Program Managers ...................................................... 129 Figure 19: Use of Expert Opinion, SMEs, or Authorities........................................................... 130 Figure 20: Program Manager Compliance with DoD Documents.............................................. 132 Figure 21: Use of Standard Defense Decision Factors ............................................................... 132 Figure 22: Contribution of Systematic Process to Decision Making .......................................... 134 Figure 23: Completeness of Organizational Decision Processes ................................................ 137 Figure 24: Use of Criteria for Key Defense Acquisition Decision Points .................................. 139 Figure 25: Value of Emerging Technology to Acquisition ........................................................ 140 Figure 26: Program Managers' Perception of Emerging Technology Opportunities ................. 142 Figure 27: Perceptions of Systematic Process Support to Decision Making .............................. 143 Figure 28: Observations Arising from the Question 7-8-9 Comparison .................................... 146 Figure 29: Value of Emerging Technology Opportunities at Different Acquisition Phases ...... 148 Figure 30: Program Manager Valuation of Emerging Technology ............................................ 149 Figure 31: Enabling Contribution of Emerging Technology ...................................................... 150 Figure 32: Effectiveness of Organizational Level Decision Making Processes ......................... 151 Figure 33: Program Manager Perception of Emerging Technology Opportunity ...................... 153 Figure 34: Effective Decision Making in Response to Opportunity Presentation ...................... 154 Figure 35: Extent Emerging Technology Opportunities are Incorporated ................................. 155 Figure 36: Extent Program Managers Use Other Than DoD Selection Criteria ......................... 157 Figure 37: Employment of Range of Decision Considerations .................................................. 158 Figure 38: Consistent Criteria Use for Emerging Technology Decisions .................................. 160 Figure 39: Summary of Decision Factor Survey Ratings ........................................................... 164 Figure 40: Survey Results for Decision Factors Arrayed by Response Category .....................