Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture

Download Now
Provided by: Cornell University
Topic: Big Data
Format: PDF
The authors present the architecture behind Twitter's real-time related query suggestion and spelling correction service. Although these tasks have received much attention in the web search literature, the Twitter context introduces a real-time \"Twist\": after significant breaking news events, they aim to provide relevant results within minutes. This paper provides a case study illustrating the challenges of real-time data processing in the era of \"Big data\". They tell the story of how their system was built twice: their first implementation was built on a typical Hadoop-based analytics stack, but was later replaced because it did not meet the latency requirements necessary to generate meaningful real-time results.
Download Now

Find By Topic