Using Latent Semantic Analysis of Email to Detect Change in Social Groups

Email text data is a rich resource that, when properly used, may enhance warning of economically material events in commercial enterprises. Armed with the temporal text classification of an email time stamp on a large data set, the authors combine latent semantic analysis with statistical process control, to detect potential change in sentiment. A novel approach to identifying the causes of change is proposed by averaging concept scores across statistically significant time periods and then performing an inverse singular valued decomposition. The resulting term by significant time period matrix, is used to explore potential causes of change and are compared to historical events.