Penetration testing—the need for it and how to go about it—has long been a source of debate among IT leaders. Should you expose your system to attacks from outside your network boundaries? If you do, how do you protect yourself from the possible fallout?

Still, for many IT leaders, penetration testing allows them to see their system vulnerabilities and to take measures to fix them. The onus of how to go about implementing penetration testing usually falls on the shoulders of the IT manager.

Recently, we asked TechRepublic members to share with us their stories of implementing penetration testing at their organizations. Here we’ll present two interesting accounts, one from an IT manager who used an internal testing strategy and the other from a security manager who outsourced it.

Using internal resources
Don Willadsen, an IT manager for the United States Army, was a network administrator when he was tasked to do penetration testing on the network. The internal employees whose task it was to do the testing were called the “Red Team.”

“We asked the Red Team to come play on our network over a one-month period, during which we were already working 24/7 shifts and could watch the network day and night,” Willadsen said.

The Red Team began by gaining physical access to the inside of the network and doing pingfloods to map the network. Because the Red Team had bragged to someone in the building about who it was, Willadsen was able to find out about the Red Team’s arrival a few minutes before the pingflood began. As soon as the pingflood started, Willadsen pulled the plug on that subnet and sent an intrusion report, but the Red Team already had the data it needed. For his part, however, Willadsen had the host names and IP addresses of the laptops the Red Team used. The first battle was a draw.

“I think mapping the network from the inside was cheating because they were allowed to gain physical access with their laptops and software tools, some of which (such as ISS Internet Scanner) the average employee couldn’t possibly obtain. However, some script kiddie tools could also map the network from the inside, so it may have been allowed by the Rules of Engagement,” Willadsen said.

Willadsen detected nine attempts to access the network and reported them as required. The Red Team also made some attacks that never reached the network, and were therefore not detected, because of access control lists on the border router. Only the DoS attacks were successful, and those only temporarily. The ISS RealSecure stopped most of the attack forms, and once an attack was detected and verified, Willadsen’s staff was able to prevent further attacks by taking three minutes to reconfigure the border router to exclude that IP address.

“In the After Action Review, the Red Team revealed that it was responsible for seven of the attacks, with two attacks being real-world. My team had two advantages over the Red Team: We had a later version of ISS software than the Red Team did, so its Internet Scanner was of no use, and two of our team members were trained in Atlanta by ISS to use the products, whereas the Red Team was not. The Red Team learned as much from the exercise as we did, hopefully, and has gotten better since then,” Willadsen said.

Hiring outside testers
A security advisor (SA) for a bank in the Middle East, who wishes to remain anonymous, submitted our second story of penetration testing.

During an initial survey of his organization’s security baseline, it was quickly apparent to the SA that a security review of the networks, OS, and applications had never taken place.

He met with mentalities typical of many IT organizations, namely: “We have a firewall protecting our external vulnerabilities, therefore we’re protected, aren’t we,” and “people have to be inside our organization and connected to our networks to do any damage, don’t they?”

“It took some time to convince them, through demonstrations, that their systems were vulnerable—both inside and outside—from the unknown number of modems and external links, misconfigured routers (including those of our ISPs—where it was child’s play to conduct a redirect and sever all communications), to unpatched servers, weak password rules, little housekeeping of users accounts, etc.,” said the SA.

Having convinced senior management of the requirement to conduct full, internal, and external penetration testing, the challenge was then to find the right people to do the job. The staff didn’t, at the time, have the right skills or experience to conduct such work. The IT department needed to have someone who could think like a hacker, but be bound by legal confidentiality clauses and laws.

“After some investigation, I eventually found a company who really understood what penetration testing was, and would conduct it to meet our needs. (This company listened to the customer instead of just insisting on their own way of doing things.) This company not only had a well-qualified staff, they had a full laboratory to conduct external testing and they were open to site visits and reviews of their work. They assisted in the building of an excellent contract and work plan. So many other companies we looked at could not do this, and would not all allow visits—primarily because they were working out of just about anywhere they could connect a computer,” the SA said.

The contract
In the interests of both companies and the customers involved, the contract was reviewed and endorsed by lawyers. The contract was not built on just a one-off. The testing company built a series of tests over time; ensuring that appropriate pressure was applied to correct the findings and recommendations before the next phase of testing commenced. This proved to be the right approach for this particular organization. The SA cautions that when writing a contract, you should be sure to factor in sufficient time between the first and second tests for corrective actions to be adequately implemented—people have to understand why changes were required, and then IT has to study how to implement them.

Because such testing had never been completed before (and many people within the organization didn’t understand what was involved), the SA built into the contract an element of on-the-job training. One of the objectives of this was being able to train his own staff and educate the administrators about penetration testing, so that in the future, they could help themselves better.

“It was not, is still not, my intention that we will do all penetration testing ourselves, because I believe that our staff would never have the refined skills, equipment, or time to do the job properly. However, our daily awareness and risk analysis has increased immeasurably with ongoing monitoring,” said the SA.

The debate
The SA admits to thinking long and hard over the ethical debate of bringing in “outsiders” to conduct penetration testing, but knows that he made the right decision. “For us, I can say that penetration testing worked. It would not have been successful if we had relied solely upon internal resources because they didn’t have the time or required skills. In using the third party, we quickly expanded our knowledge and skills, obtained more tools to assist our daily monitoring and awareness, and improved both policies and procedures to address such important issues. We’ve trained and developed a number of our own staff, and we now conduct almost constant testing of new or changing servers, OS, applications, and infrastructure.”

Penetration testing isn’t easy, but these members feel that they’ve learned a great deal and that the benefits are worth the effort.