An attempt to explain the general security benefits of open source security by way of discussing only a single factor in a system's security will tend to be deficient. Chad Perrin urges skepticism for all security models when choosing software.
Discussions of the relative security benefits of an open source development model -- like comparative discussions in any realm -- all too often revolve around only one factor at a time. Such discussions tend to get so caught up in their own intricacies that by their ends nobody is looking at the big picture any longer, and any value such discussions might have had has already evaporated.
When trying to engage in a truly productive exchange of ideas, it is helpful to keep in mind the fact that when something is worth doing, it is usually worth doing for more than one reason. This applies to the security benefits of an open source development model, as it does to other topics of discussion. A small number of such factors behind the security benefits of open source development are examined here:
The Many Eyes Theory
Probably the most common and obvious scratching post in online discussions of open source security is the so-called "many eyes" theory of software security. The simple version is often articulated by the statement that given enough eyeballs, all bugs are shallow. The most common retort is that open source means that more malicious eyeballs can see your bugs, too.
Of course, this counterargument is predicated upon a generally false assumption, that bugs are typically found by looking at source code. The truth is that bugs are found by mistreating software and observing how it fails, by reverse engineering it, and by a user simply going about his or her business until discovering that a program has done something like delete all of the previous hour's work.
This theory of improved security is no true guarantee of practical security benefits, even if the most common counterarguments against it are mostly full of hot air, though. Possibly the most difficult counterargument to dismiss effectively, despite its fallacious reasoning, is the simple statement that the open source "many eyes" theory of software security does not work because it provides no guarantees. It is difficult to dismiss because it is true that no such guarantee exists. That difficulty is awfully frustrating because many people who make such arguments, and presumably many of those who listen to them, completely overlook the fact that it does not have to be a guarantee to be a benefit. All it needs is to be an increased likelihood of security, or even just increased opportunity without a counterbalancing problem.
The "Not Microsoft" Theory
Microsoft is widely recognized as a symbol of poor software security. Generations of computer users have essentially grown up experiencing the security issues that make such a reputation so well deserved. The fact that MS Windows 95, 98, and ME all failed to even do something so simple as maintain memory space separation is the kind of gross, dangerous oversight in the design of a system that can permanently tarnish a reputation. The simple fact that your software does not come from Microsoft lends it an air of at least a little legitimacy amongst some people, because while that does not prove it is secure, it at least suggests it may not share the traditional problems of MS software security.
Microsoft has launched a number of initiatives over recent years to try to rehabilitate that reputation, of course. Its successes in this area are owed to the fact that more money has been spent advertising a greater focus on security than on any actual security focusing efforts themselves, but meaningful changes have been made in the way Microsoft produces software in attempts to improve the technical security of that software in addition to the copious quantities of marketing dollars spent on apparent security. These days it is, for many people, not sufficient for purposes of making people think your software is secure to merely say, "This is not software from Microsoft." If you want to impress people, you have to explain how it is secure, and not merely that it is not software from some vendor well-known for its past security failings.
Even so, pointing out that Microsoft was not involved in your software development process can still carry some weight with at least some readers or listeners. Microsoft is still going through some growing pains on its way toward producing more secure software, and internal conflicts between secure design and other (less technical) concerns for the commercial viability of its software offerings still present major hurdles to improving software security. Just be aware that to effectively use this argument you will probably need to be able to back it up with current, relevant explanations of the security problems that still lurk in the software development processes of this industry giant.
The Transparency Theory
Possibly the most unassailable security argument for open source software development is that of transparency. Because the source code is open, and because (especially in the case of very popular projects) many people are motivated to sift through the source code of open source software projects for a variety of reasons, that source code is likely to be seen by a great many people. Apart from the notion that bugs become "shallow" when enough eyeballs scrutinize the software, those eyeballs also provide some discouragement for those who might try to sneak malicious -- or at least dubious -- functionality into the design of a software system.
The most obvious and immediate counterexample is probably the OpenBSD project's 2010 scandal over a claim that its IPsec implementation contained an FBI "backdoor". The fact of the matter is that this claim is most likely false, whether the person making the claim knows it or not; a number of developers have set out to analyze the design of the system and find such backdoors if they exist, and come up empty-handed. Even if the claim proved true, however, it would not invalidate this theory of improved security for open source software.
The fact of the matter is that the quick announcement of the claim by the OpenBSD project founder, Theo de Raadt, illustrated the effects of open source software development as a motivator for being honest and up-front with the public about security matters. By contrast, the majority of large corporate software vendors would have been more inclined to sweep such claims under the rug and, even if they proved true, try to keep such knowledge out of the hands of users for fear it might affect sales. There is little motivation to share such issues when it might damage sales figures in cases where the closed source development process (and development employees who have signed NDAs) ensures a very low likelihood of outsiders stumbling across such vulnerabilities independently.
The Unix Theory
The Unix style of operating system (and other software) design provides substantial benefits for security over many other approaches to software design. Basic (but complete) privilege separation, modularity, and decades of testing under fire are among the many reasons Unix-like operating systems often provide greater security benefits than competing OSes.
While this argument stands up well for certain specific pieces of software or user environments, it is not universally applicable. Open source operating systems like Haiku and Plan 9 are not very Unix-like and, while they may be very well designed systems with strong security characteristics, discussing the security benefits of Unix does not address these systems' benefits as open source software. More to the point, there are closed source Unix-like systems that offer much the same benefits. Some other open source software is also not very Unix-like, such as the Mozilla Firefox browser and the Pidgin multiprotocol IM client, both of which take a monolithic, "feature rich" approach to software design that stands in marked contrast to the Unix approach of designing programs to do one thing, do it well, and interface easily with other programs that do other things.
For those pieces of open source software that do conform to the expectations of Unix, however, this argument is alive and well, and quite valid. The extent to which tools like
grep have grown out of control in some implementations and drifted away from the Unix philosophy of software design is troubling to some, but the tenets of that philosophy are still visible in the basic design of these tools. Simplicity, clarity, and care in the design of software is a pleasant benefit that arises in part from such an approach to software development.
Breadth of knowledge
The important thing in considering such matters is to be aware that circumstances are more complex than a single, pithy statement about the security of open source software. Several arguments are relevant to discussions of the security benefits of open source development, including not only those listed above but others as well. Do not neglect all but one, and get yourself backed into the dead-end of a merely semantic argument relating to that one single security benefit of open source software development. Do not put all your eggs in that single basket when selecting software for your use, either. Seek out, and consider, other potential arguments, not only for discussions with others who might disagree with your analysis, but also because you need to know something about the major arguments to make an informed decision about what software to use and how to use it in the most secure manner.
Finally, do not make the mistake of making -- or being taken in by -- the Invulnerability Theory. Some have claimed that certain open source software, especially including Linux in general or Ubuntu Linux in particular, is impervious to security exploits of any kind. Such claims are patently false, and in fact quite obviously ridiculous. Linux is not the most secure operating system, and neither is anything else, regardless of development model.