A Focused Crawler for Dark Web Forums

The unprecedented growth of the Internet has propagated the escalation of the Dark Web, the problematic facet of the web associated with cybercrime, hate, and extremism. Despite the need for tools to collect and analyze Dark Web forums, the covert nature of this part of the Internet makes traditional web crawling techniques insufficient for capturing such content. In this paper the authors propose a novel crawling system designed to collect Dark Web forum content. The system uses a human-assisted accessibility approach to gain access to Dark Web forums. Several URL ordering features and technique enable efficient extraction of forum postings.

Provided by: University of Arizona Topic: Software Date Added: Dec 2010 Format: PDF

Find By Topic