A big data computational framework for enterprise level statistical process monitoring

Doctor of PhilosophyDepartment of Industrial & Manufacturing Systems EngineeringShing I. ChangThe emergence of big data storage together with the evolution of sensor technologies has expanded the amount of data that complex manufacturing facilities can produce. Almost all process variables in the factory can be measured and the data can be stored in data lakes in cloud servers. This big data phenomenon has presented challenges and opportunities for quality improvement teams. While the traditional control charts are still widely used, they are often isolated tools for monitoring product quality characteristics scattered in a manufacturing system. The need to monitor full systems becomes even more pressing with the emergence of smart factories in the next industrial revolution called Industry 4.0. The goal of this research is to develop a big data computational framework for enterprise-level process monitoring that tracks different variables simultaneously and provides near-time system status updates. To achieve this goal, a novel methodology called Technique of Uniformally Formatted Frequencies (TUFF) is developed that standardizes continuous, discrete and profile variables into comparable statistics, classifies these statistics into four colors using ideas from pre-control charts and summarizes these colors to a single frequency table. This table is used to compare the current situation to historic data and to decide if the performance of the system has changed. A higher resolution of the results identifies the temporal and spatial location of possible change. The comprehensive monitoring method uses all the available data and monitors both quality characteristics as well as process parameters near-time. Additionally, the method is easily scalable to handle big data level datasets. Extensive simulation studies identify the sensitivity and other characteristics of the TUFF method. This dissertation also redefines one of the more popular Six Sigma continuous improvement methods of DMAIC (Define, Measure, Analyze, Improve, and Control) for the manufacturing environment. The redefined method is Measure, Define, Analyze, Improve and Control (MDAIC), where the unit in need of improvement is identified automatically by the data. The research integrates the TUFF statistical system monitoring method to the MDAIC framework and provides a solution for the implementation of the method in a big data environment based on the MapReduce algorith