Non-intrusive, Automated Log Discovery and Parsing

Todayâ s distributed systems generate log data for tasks such as debugging failures. When aggregated into a log analytics tool, this data can be queried to perform anomaly detection and failure diagnosis at a scale that is infeasible otherwise. However, before analysis, this data must be discovered and parsed. harvester is a tool capable of automatically finding and parsing logs for indexing. harvester â s approach is novel in that it generates regular expressions for parsing using the constants of the executables that authored the messages. This allows harvester to parse logs for any software, without source code, so long as constants can be extracted from the softwareâ s binary. In our evaluation of two distributed systems written in Java and C, we are able to achieve an accuracy of 67.7% for the generated expressions. Our inaccuracies are remediable with slightly improved logging practices without having to overhaul existing logging.M.A.S