Every parcel of digital information–including the email you send, Skype calls you make, and websites you load–is transmitted across the web in a formatted piece of structured data known as a “packet.” Inside this packet is structured metadata that assures your data is routed to the proper destination. Analyzing these packets is a process known as deep packet inspection (DPI), and the practice is employed daily by enterprise companies, internet service providers (ISPs), and media companies.

TechRepublic’s smart person’s guide is a routinely updated “living” precis loaded with up-to-date information about how deep packet inspection works, who it affects, and why it’s important.

SEE: All of TechRepublic’s smart person’s guides

Executive summary

  • What is deep packet inspection? A network packet is a formatted and discreet unit of data. Deep packet inspection is a method of analysis that dissects network data to extract useful metadata.
  • Why deep packet inspection matters: Deep packet inspection illuminates network trends, helps ISPs optimize bandwidth and throughput, and can reveal user behavior.
  • Who deep packet inspection affects: Because deep packet inspection inherently involves exposing sensitive data, IT departments at enterprise companies, ISPs, and consumers are most impacted by the practice.
  • When deep packet inspection is happening: Deep packet inspection has been a useful IT tool for nearly two decades. As the internet evolves to include mobile and IoT devices, deep packet inspection is being used more and more frequently.
  • How to get started with deep packet inspection: First, learn more about the process. Next, search GitHub and other code repositories for open source tools. Finally, speak with your IT department to learn more about how your company performs deep packet inspection on a daily basis.

SEE: Cybersecurity in 2017: A roundup of predictions (Tech Pro Research)

What is deep packet inspection?

Internet traffic is composed of small bundles of data known as packets. Packets wrap digital information in a cocoon of metadata that identifies traffic source, destination, content, and other pieces of valuable details. Analyzing digital traffic is a lot like analyzing automobile traffic: Patterns reveal useful insights. By studying metadata like headers using deep packet inspection (DPI) network specialists can learn how best to optimize servers to reduce overhead, detect and deter hackers, combat malware, and glean intimate details about user behavior.

Although DPI has a number of uses, the practice is rooted in enterprise network security. Sniffing traffic in and out of a network is understandably useful for preventing and detecting intrusions. Detecting and blocking the IP of malicious traffic is particularly effective at fending off buffer overflow and DDoS attacks.

DPI is also used by internet service providers. If packets are mail, ISPs are the postal service and have access to unencrypted web traffic as well as packet metadata like headers. This provides ISPs with an abundance of useful information, and the companies leverage access to user data in a number of ways. Most ISPs in the United States are allowed to turn user data over to law enforcement agencies. Additionally, many ISPs use consumer data to target advertising, analyze file sharing habits, and tier access service and speeds.

Additional resources:

SEE: Three ways encryption can safeguard your cloud files (Tech Pro Research)

Why deep packet inspection matters

Although packet-sniffing is an archaic tactic, due to the sheer scale of connected devices DPI is more relevant today than in prior epochs. DPI is germane for three primary reasons:

  • The scale of connectivity. The internet today, particularly mobile, is more important now to more people for more reasons than ever before. Every company and organization relies on network inspection technology to optimize traffic, reduce overhead, and fend off cyberattacks. DPI isn’t the only line of defense, but for many organizations, scanning and analyzing packets is the first line of defense.
  • The burgeoning IoT market. Like the mobile market before it, IoT inherently means millions of additional devices will come online in the next few years. Many contemporary IoT devices often lack standard firmware and security standards that could protect the devices from being lassoed into a zombie botnet. DPI will shield ISPs and networks from IoT DDoS attacks and help security analysts learn more about critical IoT security flaws.
  • Privacy concerns are mainstream. DPI helps media companies learn about customers in ways unimaginable 10 years ago. Every page you load and every piece of communication you send is filtered and routed through an ISP. No longer “dumb pipes,” internet service providers are vertically integrating with media companies (the Comcast NBCUniversal and AT&T Time Warner mergers are two examples) and leveraging their data to target consumers with advertising and assist law enforcement agencies with intelligence gathering.

Additional resources:

SEE: Interview with a hacker: S1ege from Ghost Squad Hackers (TechRepublic)

Who deep packet inspection affects

Beyond enterprise and SMB companies, DPI is used primarily by:

  • Media companies. Media companies have a storied history of consolidation. When ISPs buy media companies they combine broadcast data with digital data to determine everything from television and web programming to corporate and consumer internet service prices.
  • Law enforcement agencies. It is legal, and sometimes required, that ISPs gather and share DPI-gathered data for crimes involving intellectual property violation and drug and human trafficking.
  • Consumers. Most consumers are aware that, love it or lump it, personal data is for sale. Most consumers are likely unaware that their ISP is probably analyzing, anonymizing, and reselling personal browsing data to advertising companies.

Additional resources:

SEE: New World Hackers group claims responsibility for internet disruption (CBS News)

When deep packet inspection is happening

Deep packet inspection, known also as full packet inspection or data packet inspection, dates back to the ARPAnet. The ARPAnet predated today’s internet and was the first computer network to use the TCP/IP data transfer protocol. Managing proto-packets helped engineers learn how to use header and metadata information to mitigate security challenges related to UNIX.

ARPAnet went dark in 1990, but as the modern internet went mainstream so too did TCP/IP challenges. A model called Open Systems Interconnect (OSI) was developed by network engineers in the 1980s to standardize metadata congealed by the mid-1990s. By formalizing packet metadata tiers, OSI allowed for a variety of statistical analysis. For example, secondary headers, known as stateful or shallow data, allow information to be properly routed but cut bandwidth.

Tiered packet metadata also meant ISPs could more easily discriminate against types of data. With the boom of Web 2.0 and mobile in the early 2000s, ISPs realized deep packets could inspire new business models. Net neutrality has been a hot topic for nearly two decades, and deep packet inspection technology has transformed pipe owners in to data owners.

Additional resources:

SEE: Five ways small companies can get ahead through technology sharing (Tech Pro Research)

How to get started with deep packet inspection

Step 1: Read and learn. DPI is a time-tested tactic, but the IT industry is innovating quickly. The hottest trend in DPI is dedicated hardware. Cisco and several other networking manufacturers have created routers that specialize in packet sniffing and network awareness.

Step 2: A number of free, open source tools exist to help customize DPI for your organizational needs. GitHub hosts a number of free deep packet inspection tools. nDPI is an open source tool that supports a laundry list of customizable plugins.

Step 3: Speak with your IT legal department. DPI is a fundamental component of IT, and experienced network managers should be able to familiarize you with how your company gathers data. If you work in IT, make sure to speak with your legal or Standards and Practices department for usage guidance. Though it’s often legal to gather customer data, the industry is heavily regulated. DPI can be a powerful tool. Never break the law, violate company policy, or behave unethically.

Additional resources:

Read more

Subscribe to the Cybersecurity Insider Newsletter

Strengthen your organization's IT security defenses by keeping abreast of the latest cybersecurity news, solutions, and best practices. Delivered every Monday, Tuesday and Thursday

Subscribe to the Cybersecurity Insider Newsletter

Strengthen your organization's IT security defenses by keeping abreast of the latest cybersecurity news, solutions, and best practices. Delivered every Monday, Tuesday and Thursday