Enhancing Twitter Data Analysis with Simple Semantic Filtering: Example in Tracking Influenza-Like Illnesses
Systems that exploit publicly available user generated content such as Twitter messages have been successful in tracking seasonal influenza. The authors developed a novel filtering method for Influenza-Like-Ilnesses (ILI)-related messages using 587 million messages from Twitter micro-blogs. They first filtered messages based on syndrome keywords from the BioCaster Ontology, an extant knowledge model of laymen's terms. They then filtered the messages according to semantic features such as negation, hashtags, emoticons, humor and geography. The data covered 36 weeks for the US 2009 influenza season from 30th August 2009 to 8th May 2010. Results showed that their system achieved the highest Pearson correlation coefficient of 98.46% (p-value<2.2e-16), an improvement of 3.98% over the previous state-of-the-art method.