Analyzing Highway Safety Datasets: Simplifying Statistical Analyses from Sparse to Big Data

University Transportation Centers Programhttp://hdl.handle.net/10919/951712019PDFTech ReportLord, DominiqueGeedipally, Srinivas ReddyGuo, FengJahangiri, ArashShirazi, MohammadaliMao, HuiyingDeng, XinweiSafety through Disruption (Safe-D) University Transportation Center (UTC)Texas A&M Transportation InstituteVirginia Tech Transportation InstituteSan Diego State UniversityUnited States. Department of Transportation. University Transportation Centers (UTC) ProgramUnited States. Department of Transportation. Office of the Assistant Secretary for Research and TechnologySafety through Disruption (Safe-D) University Transportation Center (UTC)United StatesCrash dataData analysisData filesData qualityHighway safetyStatistical analysisSafetyCluster analysisBig DataSparse DataHeuristics MethodFinite Sample Bias AdjustmentAggregated DataDisaggregated Data01-00169A3551747115/Project 01-001Final ReportData used for safety analyses have characteristics that are not found in other disciplines. In this research, we examine three characteristics that can negatively influence the outcome of these safety analyses: (1) crash data with many zero observations; (2) the rare occurrence of crash events (not necessarily related to many zero observations); and (3) big datasets. These characteristics can lead to biased results if inappropriate analysis tools are used. The objectives of this study are to simplify the analysis of highway safety data and develop guidelines and analysis tools for handling these unique characteristics. The research provides guidelines on when to aggregate data over time and space to reduce the number of zero observations; uses heuristics for selecting statistical models; proposes a bias adjustment method for improving the estimation of risk factors; develops a decision-adjusted modeling framework for predicting risk; and shows how cluster analyses can be used to extract relevant information from big data. The guidelines and tools were developed using simulation and observed datasets. Examples are provided to illustrate the guidelines and tools.131