Carousel: Scalable Logging for Intrusion Prevention Systems
The authors address the problem of collecting unique items in a large stream of information in the context of Intrusion Prevention Systems (IPSs). IPSs detect attacks at gigabit speeds and must log infected source IP addresses for remediation or forensics. An attack with millions of infected sources can result in hundreds of millions of log records when counting duplicates. If logging speeds are much slower than packet arrival rates and memory in the IPS is limited, scalable logging is a technical challenge. After showing that na¨ýve approaches will not suffice, they solve the problem with a new algorithm they call Carousel.