USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13) 127 Juggling the Jigsaw: Towards Automated Problem Inference from Network Trouble Tickets

This paper presents NetSieve, a system that aims to do automated problem inference from network trouble tick-ets. Network trouble tickets are diaries comprising fixed fields and free-form text written by operators to docu-ment the steps while troubleshooting a problem. Un-fortunately, while tickets carry valuable information for network management, analyzing them to do problem in-ference is extremely difficult—fixed fields are often in-accurate or incomplete, and the free-form text is mostly written in natural language. This paper takes a practical step towards automati-cally analyzing natural language text in network tick-ets to infer the problem symptoms, troubleshooting ac-tivities and resolution actions. Our system, NetSieve, combines statistical natural language processing (NLP), knowledge representation, and ontology modeling to achieve these goals. To cope with ambiguity in free-form text, NetSieve leverages learning from human guidance to improve its inference accuracy. We evaluate NetSieve on 10K+ tickets from a large cloud provider, and com-pare its accuracy using (a) an expert review, (b) a study with operators, and (c) vendor data that tracks device re-placement and repairs. Our results show that NetSieve achieves 89%-100 % accuracy and its inference output is useful to learn global problem trends. We have used NetSieve in several key network operations: analyzing device failure trends, understanding why network redun-dancy fails, and identifying device problem symptoms.