System Problem Detection by Mining Console Logs
The console logs generated by an application contain information that the developers believed would be useful in debugging or monitoring the application. Despite the ubiquity and large size of these logs, they are rarely exploited because they are not readily machine-parsable. The authors propose a fully automatic methodology for mining console logs using a combination of program analysis, information retrieval, data mining, and machine learning techniques. They use source code analysis to understand the structures from the console logs. They then extract features, such as execution traces, from logs and use data mining and machine learning methods to detect problems.