Imagine that one afternoon, you receive an E-mail alert from your honeypot indicating potentially malicious activity. What do you do? Initially, you would probably take a quick look at the honeypot's logs and determine the type of exploit that is being performed and then take steps to close the hole that allowed the exploit to occur in the first place. After closing the security hole and stopping the attack, some administrators choose to call it a day. However, it may be worth your while to perform some additional forensics on the attack. In this article, I will explain why this additional investigation is important, and how to go about analyzing your honeypot's logs.
After the attack
After saving the day, you should take a closer look at your honeypot's logs. One of the first things that you would probably look at is the source of the attack. Expecting to find a random IP address, you check the source address only to find that the attack came from one of the machines on your own network. Now you've got a serious problem on your hands. One of three things has happened.
One possibility is that an employee within your company is trying to hack your network. This certainly isn't a far fetched possibility. I have seen statistics indicating that 85% of security breaches are performed by a trusted employee.
Another possibility is that the machine identified by the honeypot might be infected by a virus or a Trojan. If this were the case, then you would need to figure out what the machine is infected with and where the infection came from. Did the Trojan arrive as an E-mail attachment, or did the employee bring an infected file into the network from another source? You will also have to figure out whether the infection is isolated to this one machine or if other machines are infected. Finally, you need to figure out what the Trojan is designed to do. This is important because you need to figure out if your servers are at risk and if any sensitive information has been altered or leaked.
The third possibility is the scariest. That possibility is that a hacker has gained control over one of the machines on your network and is using it as a platform for attacking other points on your network. The reason why this is the scariest possibility is because without further investigation, you have no way of knowing whether or not other machines have been compromised or what the hacker has done to the machine that they have taken over. You will also need to do some work to find out how the hacker got into your network in the first place. Think about it for a second. If the attack against your honeypot came from a machine on your local network and a hacker was involved, then it means that the hacker has already managed to penetrate your network's perimeter. Any activity detected by the honeypot occurred after the initial penetration.
Preparing for a detailed analysis
If any of the situations that I described above were to come true, then you have got a serious problem on your hands, and you need to learn more about the attack. The problem is that the attack has already occurred and the only thing that you've got to go on is what ever information is stored within the honeypot. Depending on what type of honeypot you're using, this information could be quite volatile. Log entries might be overwritten by newer logs, buffers of captured network traffic could be flushed to make rum for new captures. Even if your honeypot could keep records forever, you never know when a hacker could plant some sort of script that will erase all of your honeypot's logs in order to conceal the crime.
The point is that when it comes to forensic analysis, the clock is working against you. If you are serious about preserving a record of the attack, then you should immediately disconnect the honeypot's network cable, followed by the power cable.
Obviously, pulling the network cable prevents any type of network based access to the honeypot. It is important that you disconnect both cables and not just shut down the machine. If you were to just shut down the machine, then it's possible that the machine could be booted via the wake on LAN feature.
You will notice that I said that you should disconnect the machine's power cable, not that you should shut the machine down. The reason for this is that if the machine is running Windows, the shut down process could potentially delete the contents of the machine's virtual memory file. I don't recommend holding down on the power button to shut the machine down either. On newer machines, pressing the power button initiates the same shut down sequence that would be run if you used the Windows Start, Shutdown option. Sure, the machine would shut down after about five seconds or so, but during the five seconds that the machine is still running, Windows is performing shutdown related clean up work. You don't want anything to be cleaned up, so just yanking out the power cord is your best option.
I should point out that none of this applies if you are running the honeypot within a virtual machine. You can easily take down a virtual machine without unplugging the computer. This article assumes that virtual machines are not being used.
Once the machine has been unplugged, you've got two things that you need to accomplish. First, you need to get your honeypot back online so that you aren't left unprotected. Second, you need to be able to safely analyze the data that you have collected without destroying it in the process.
The best way that I can think of to accomplish this goal is to remove the honeypot's hard drive and attach the drive to another computer as a slave drive. Since the computer you are using has its own hard drive and its own operating system, you can boot the machine without fear of messing up the honeypot data.
Once the machine has booted, I recommend using an imaging program to create an image of the honeypot's drive. You can then copy the image to a new hard drive and put the new drive into the honeypot machine. After doing so, boot the honeypot off of the new drive. The honeypot shouldn't care that it is running off of a different hard drive. You should be able to boot the honeypot in the usual way and resume operations. Meanwhile, you've got the honeypot's original hard drive connected to a machine that you can use for analysis. You've also got the image file that you created as a backup.
Who is to blame?
OK, so you have determined that the activity detected by your honeypot came from either a Trojan, a malicious user, or from a hacker. The next step in the process would be to narrow down the identity of the culprit. This means figuring out if the attack was carried out by a person or by an automated script. You can usually figure this out by looking at the activity logged by the honeypot.
For example, suppose that your honeypot was masquerading as an IIS server. The person or script that attacked it would usually use multiple commands during the attack. The honeypot's logs should be able to tell you what those commands were and more importantly, when they occurred. You can then study the commands and look for trends.
Your honeypot logs aren't going to directly tell you whether an attack came from a person or from a Trojan, but there are clues hidden within the logs. The following table lists characteristics of a human attack VS. an automated attack:
- Random amounts of time between commands
- Commands may contain misspellings
- Commands seem to be targeted directly at your system using previously gained knowledge of your network
- Commands are issued in quick succession (too quick to be typed)
- Commands are usually typed perfectly
- Commands may seem poorly directed (for example targeting SQL Server exploits at a server that doesn't have SQL Server installed)
Of course none of these factors absolutely guarantee that an attack was carried out by a human or by a script. You can however, usually use multiple characteristics to determine the nature of the attack. I should point out though, that an attack can be performed by a human who is running scripts.
It's not always that simple
There are a lot of different types of honeypots out there, and not all of them so clearly define an attack signature, the source of the attack, and the commands used in the attack. Some honeypots act as nothing more than glorified packet sniffers. This means that you may have your work cut out for you when trying to analyze an attack.
As if this type of honeypot didn't make analysis tough enough, more sophisticated hackers use stealth techniques to avoid intrusion detection systems. For example, one of the most basic types of activity that hackers perform is a port scan. Most intrusion detection systems define a port scan as a particular IP address trying to communicate across a certain number of ports within a specific length of time.
To circumvent detection systems, hackers may scan one port per hour, or they might use a different IP address to scan each port. My point is that if your honeypot requires you to manually analyze packets then you need to be aware that different parts of the attack can come from different IP addresses. Usually the addresses will be very similar to each other though (possibly sequential).
If you find yourself having to wade through a bunch of captured packets, then I recommend beginning the process by filtering out all of the empty packets. This could significantly reduce the number of packets that you have to look through. Next, you might try checking to see which IP addresses produced the most traffic and focus on packets from that source. You might also try looking for larger packets or packets that match a known attack signature. Once you have some packets to focus on, you can scan the packets for any readable text, so that you can start to analyze the attack. You should be able to read all of the hacker's commands unless the packets were encrypted. Unfortunately, more and more hackers are starting to encrypt packets in an effort to fool intrusion detection systems.
Hacking the hacker
After you have gathered some information on the hacker, you probably want to find out a little more about them. Some honeypots will automatically determine the domain name that is associated with the IP address that was used in the hack. Since IP addresses are geographic in nature, you can use a utility such as NeoTrace to determine the hacker's approximate location. NeoTrace is a graphical trace route utility that can determine what city someone with a particular IP address is located in.
Since you've got the hacker's IP address, you could start doing a little reconnaissance work of your own. If you aren't worried about the hacker knowing that you are on to them, you could use a utility such as Xprobe to fingerprint the hacker's machine. There are a lot of great fingerprinting programs out there and you can get all sorts of good information through them, such as what operating system the hacker is using. There are a couple of problems with fingerprinting though.
The first problem with fingerprinting is that most of the available fingerprinting utilities will perform a port scan as a part of the fingerprinting process. It will be an instant tip off to the hacker that you are on to them if they receive a port scan from the IP address (or domain) that they just hacked. If you don't want to tip the hacker off, then you might consider using a fingerprinting utility called P0F.Â I have personally never use P0F, but according to some of the hacker Websites, P0F has the dubious distinction of being able to fingerprint a machine without making its presence known.
It does this by monitoring the traffic flowing out of the target machine rather than interacting directly with the machine. Supposedly, P0F can determine the hacker's operating system, how the hacker connects to the Internet, who the hacker's ISP is, and the program is supposedly not fooled by firewalls that might stand between you and the hacker. Once you have this information, you could get a court order for the ISP to hand over the hacker's identity.
This leads us to the second problem associated with
fingerprinting. Only an amateur hacker will use their real IP address in a
hack. A skilled hacker will either spoof an IP address or take over someone
else's computer and remotely launch the attack from that computer so that it
gets traced back to someone innocent.Â
This means that the attack will probably get traced to some little, old
Taking over another computer and launching an attack from it is the preferred method even though it's easier to spoof an IP address. The reason is because it's simple to send packets from a spoofed IP address, but it's impossible to have packets sent back to you without somehow relaying them to your real address. Taking over a computer allows the hacker to safely use two-way traffic while obscuring their identity.
I am not condoning this technique. If you decide to use this technique, don't send me an E-mail and tell me about it, because I don't want even want to know about it. I couldn't conclude this article without sharing a rather amusing story with you though.
In the process of researching this article, I stumbled across a post on a Web site from someone who was tired of getting hacked. The person created a honeypot that was designed to look like a workstation that contained a number of hacker tools. One of the tools was a well known utility that is used for cracking passwords on certain adult Web sites. The thing was though that the file on the server wasn't really the cracking program that it claimed to be. It was actually a copy of Back Orifice in disguise! The idea was that if the hacker ever downloaded and executed this program, they would open a back door into their own system and the victim could take revenge against the hacker. I'm not sure what the outcome of this was, but I had to pass that one along.