Big Data

System Problem Detection by Mining Console Logs

Date Added: Aug 2010
Format: PDF

The console logs generated by an application contain information that the developers believed would be useful in debugging or monitoring the application. Despite the ubiquity and large size of these logs, they are rarely exploited because they are not readily machine-parsable. The authors propose a fully automatic methodology for mining console logs using a combination of program analysis, information retrieval, data mining, and machine learning techniques. They use source code analysis to understand the structures from the console logs. They then extract features, such as execution traces, from logs and use data mining and machine learning methods to detect problems.