Web Development

Website and app security tips for software developers

In response to a troubling security report, Justin James shares what he's learned about securing applications after writing software for almost 20 years.

WhiteHat Security recently published its 2012 Website Statistics Report (PDF), which gives a broad view of the state of website security. I spoke with Jerry Hoff, WhiteHat Security's VP of Static Code Analysis, to get an idea of how bad the problems with website security are, why we still have these problems, and what software developers can do to remedy them. Sadly, despite much improvement from previous years, there is little cause for celebration.

The report shows a dramatic decline in overall serious vulnerabilities, from 1,111 in 2007 to 230 in 2010, and now "only" 79 in 2011. While that's a fantastic drop as a percentage, the fact that the average website has 79 vulnerabilities is troubling. Even more bothersome are the types of vulnerabilities as of 2011 (in parenthesis are the percentage of sites showing these vulnerabilities): Cross-Site Scripting (55%), Cross-Site Request Forgery (19%), and the most dangerous (in my opinion) of the bunch, SQL Injection (11%). While this is a slice of the vulnerabilities found, these particular issues irk me to no end, because they should not exist in 2012; proper Web frameworks, tools, and libraries do a great job of mitigating these problems. When I see an application or website with these problems, I know there is a developer behind the scenes hand-coding things, usually for no good reason, and clearly without the proper skills or knowledge to be hand-coding this kind of stuff. I find SQL Injection vulnerabilities particularly offensive, because in this day and age, it is substantially more work to create code that is vulnerable to them than code that is protected.

Mr. Hoff and I talked quite a bit about the underlying reasons for these problems. Something that we definitely agreed on is that a lot of developers do not know how to address security concerns. There is really nothing new here, we've been seeing this problem for a long time and just throwing more and more technology at the problem to compensate. For example, in the WhiteHat Security report, 71% of the found vulnerabilities can be plugged with an application firewall. But think about that for a moment... 71% of the vulnerabilities are so commonplace that an automated rules-based tool can zap them. This tells me that right off the bat, 71% of the problems should not exist in the first place.

The heart of the problem is that the amount of knowledge needed to be a "competent software developer" continues to increase, while the useful lifetime of the knowledge continues to decrease. There are simply not enough hours in the day for someone to work a full-time job as a software developer and get the continuing education needed to be a competent software developer unless they are willing to sacrifice an awful lot of their personal time or work for one of the rare companies that truly supports learning. To make it even worse, being a good software developer requires at least 5 - 10 years of experience (depending on what you are doing and the environment you are doing it in) on top of initial foundational learning. So just as developers truly hit their stride, they are in prime territory for life events like marriage, children, and generally slowing down the pace of their lives. I can tell you from personal experience that in the last six years since I started writing for TechRepublic, I went from "single, childless adult with nothing to do after work except play video games and do more programming" to "married adult with two children who has watched two films in the last year that were not suitable for a child."

Several weeks ago, Dr. Dobb's released a fantastic salary survey. Skip ahead to page 9 for the worst kick in the head that you'll get all week. What the chart shows is that the average IT worker's salary peaks between 36 and 55 and decreases from there. In addition, the average salary for the 36 - 45 band and 46 - 55 band are identical. In other words, there is a massive 20 year career period where someone's value does not increase. In comparison, IT managers show a curve more like you would expect, where the salary continues to increase until the 46 - 55 band, and then shows a drop off.

There is no coincidence here. How do I know? Page 10 is the second worst kick in the head of the week: the breakout of IT workers by age. We see a decline from 35% of IT workers in the 46 - 55 age bracket to 14% in the 55+ age bracket. It is clear that developers have a shelf life, and the expiration date is around 55.

All of the education in the world is not worth a hill of beans unless you can internalize it, and that requires a lot of hard real-world lessons, the kind that only those who have a lot of experience have learned. My overall level of "in-demand skills" (i.e., the kind that you'd talk about in a job interview or put on a resume) has not gone up in the last five years and if anything it has gone down, but my base of foundational knowledge has continued to grow. The Justin James of today knows an awful lot more about safe, sane coding practices than the Justin James of 2005 when I started writing for TechRepublic, or the Justin James of 2000 when I graduated college, or the Justin James before that. In 2000, I knew a lot more than the typical Web developer, because I had started in 1995 when Perl was just starting to become the hot tech to make Web pages interactive and dynamic. By 2001, I had implemented significant parts of the HTTP protocol by hand and from scratch, and in the process and I learned a l lot about how web security works.

This brings me back to the security issue. Without a doubt, if I didn't have the nearly 20 years of experience I have had writing software, I would be churning out code full of holes. I know this, because I've seen too many less experienced developers make the same class of mistakes over and over and over again that I was making in 1997 or 2003, even when I "knew better." I know why I made those mistakes -- it was either the hubris of "I can roll my own better than off-the-shelf," or the idea that slapping something together quickly would be fine "for now" and I would pay the technical debt off later. I was wrong on both counts, every single time.

What I've learned since then, when it comes to security, is the following (hardly conclusive, and in no particular order):

  • Sanitize EVERYTHING, including data pulled from databases, files, Web services, or anything else that is not a literal expression within your code.
  • Use pre-made tools, frameworks, libraries, code components, etc. to take the burden off your shoulders. It is better to re-work your business logic to use a pre-tested piece of code than to roll-your-own and be stuck trying to get the security kinks worked out.
  • Do not box yourself into a corner with integrations so that a base system is "upgrade proof" or even "upgrade resistant." Applications get critical security patches all the time; if you cannot do what you need or want to do to a system without preventing the base system from being patched or upgraded, either you need to re-evaluate your needs or you need to use a different system. End of story.
  • Never generate SQL statements within an application. If you insist on writing your own SQL (hint: you probably can't write it any better than a code generator, unless the task is unusual or you are a SQL wizard, and most tasks are not that unusual and you probably aren't a SQL wizard), put it in stored procedures. If I ran the world, compilers would flag inline SQL as an error and refuse to compile.
  • Use the various automated tools out there to perform periodic security audits.
  • Treat security fixes as your most important features.
  • Approach your code with the mindset of an attacker. If you wanted to vandalize the system, destroy data, or implant little bombs for users, what windows are wide open and which doors are unlocked?
  • Keep learning, keep working, keep studying, and keep talking to people. Get exposure to different ideas. Find out how other people are solving problems, and investigate them!

J.Ja

Keep your engineering skills up to date by signing up for TechRepublic's free Software Engineer newsletter, delivered each Tuesday.

About

Justin James is the Lead Architect for Conigent.

9 comments
mattohare
mattohare

We've had discussions in articles and the threads after them about all of the COBOL apps that are still here to stay for a very long time. For better or worse, they work and they're too expensive to replace. I wonder how many web sites/web applications in PHP are like that. For better or for worse, they work. And, they'd be too expensive to replace. Unlike the COBOL apps, however, the website flaws are out for the public to enjoy.

Tavis
Tavis

The article appears to be interpreting Dr Dobb's chart to indicate the salary scale of an individual over their own lifetime. This is not what the data shows. The people in one age range will not become the people in the following age ranges, they are different cohorts. Projection is not possible without some caveats. In fact, the report starts off by looking at a three-year trend in salaries in various areas, which is a confounding factor. There could be many different explanations as to why salary averages (in a survey response) are less in over 55. Maybe the older age group work less hours, or have otherwise different work patterns. Maybe the younger groups are better qualified, while the older groups have more people who migrated into the sector with fewer qualifications and mixed experience.

ahanse
ahanse

If the developers of the sites that feature in this list are from credentialed sites, then what hope is there for the rest of us? Interestingly the vulnerabilities are well documented and mostly understood by experienced developers. So WHY are there so many problems occurring? Is there more to the story than what we are told? Could it be that the fore mentioned developers are just your standard developer that bounces from problem to problem with the added luxury of gaining experience on the way (maybe) at other peoples expense? Experts abound. Just look to the Internet. With the internet only ten plus years old as most would know it and programming languages for web development less still, it is no wonder we have problems. It has taken centuries of evolution to develop a business model that worked and now the same businesses open themselves up to a model less than ten years tested. Give it time!

AlphaCentauri
AlphaCentauri

1: Always use parameterized queries (never concatenate user input or for that matter any externally supplied strings to a query) 2: Use of stored procedures does not immunize you against SQL Injection attacks, make sure you use parameterized queries there also 3: Don't just replace or escape characters on the way in, you can accidentally change business meaning 4: Always escape output at the view layer. Do not do this at the model layer since the web is not the only output medium. Other views may need different output sanitization Just doing these will go a long way to fend off attacks on your software. This is by no means a complete list, but it's a good start.

mattohare
mattohare

I have had to write my code a lot of the time. That means writing just enough SQL code to maybe qualify as a wizard. (You will only find Stored Procedure calls and the odd SELECT 1 AS TheResult in my main code project, however.) I hesitated at saying to use Object Oriented at first because of all of the people that claim it to be a pipe dream, or 'a really good idea, but no one has the time.' I have a shock for you, I wouldn't have the time to develop if I didn't encapsulate into some sensible objects. In my main application, I can grow it with new functionality often by inheriting or implimenting things that I've already written. I wrote an interface early on with another project handling various data files. I get a new file type, and most of the basic code is done. I just write the bits that are different. Think about it, the best platforms for web development now follow MVC philosophy. That's a web development API in a rich object model. It's so automatic, that most developers can make a decent website with less work using one of the MVC platforms than they could by customising a content manager. Justin's right. We don't have time to hard code everything from scratch. My addition is that when you can't use a platform library, then build your own properly. Encapsulate. It just might extend your shelf life by a decade or two.

notreal
notreal

I can't see how many clients would hang around after you tell them that the business model has to change to fit the code? Most clients have an expectation that you can write, or at least source, code that will meet their objectives. Isn't there an inherent problem with code libraries; that their code is publicly available and so, once a vulnerability is found (and there will be more hackers looking...) it applies everywhere that the library item/framework is used? Quite apart from that, code libraries are the reason the same user-irritating problems are repeated all over the web ("Ooooops!!!!Did you just enter an incorrect URL?" No, actually, I clicked on one of your links...). I write my own code, and unless they get on my server, no hacker knows what that code is. I transpose punctuation and the letter 'x' to something else, transpose any SQL words then take out ALL characters that aren't A-Z, a-z or 0-9 and completely disallow Javascript phrases such as Alert and so on. It seems to me most online advice, and the code libraries, just allow far too much from the website user, when restricting folk to plain English and so not letting them input %, $ or < (etc.) will stuff up most hackers pretty neatly. I have even seen code libraries that helpfully return MySQL errors and put them on screen for the user ("Sorry, unable to connect to MySQL database "address" with password "here it is" ".

Sterling chip Camden
Sterling chip Camden

1. Pre-made components may have security vulnerabilities that you cannot know about. Make sure you use open source components from a reputable developer. 2. Sometimes SQL has to be generated on the fly. Parameterized queries are your friend.

Justin James
Justin James

I understand that the chart is not lifetime income, but a snapshot of current income. Some of the differences you cite are valid (qualifications), some are not (work hours... because nearly everyone in IT is salaried). J.Ja

Railroad Buff
Railroad Buff

... for my real name. I have a hyphenated double first name and up front two ancestors names (which I almost never use) and no middle name. If I understand you right, you'd convert Jean-Jacques Rousseau into Jean J Rousseau even though he didn't even know what the concept of a middle name is. This becomes an issue when digitized records need to be matched with such a mundane thing as a birth certificate. You're in danger here to violate your own tenet - respecting the client's business model.