Every now and then, I will see a "dispelling the myths of open source" type of article, blog, discussion, or whatever come my way, and it always seems to come around to the "more eyeballs means less defects" idea. For whatever reason, many open source proponents seem to believe that there is this rear guard of closed source folks spreading FUD about open source (even Microsoft has toned down their rhetoric lately). I think that less than 10% of the knowledgeable people out there actually claim that closed source is inherently more secure than open source. It definitely seems that most people (at least amongst those that voice an opinion) believe that open source software is inherently more secure than close source software.
In reality, what matters much more than "open" or "closed" source, is who is writing and reviewing the code, why they are doing it, and how long they are doing it. If I compare OSS project "Project A" to closed source project "Project B", and "Project A" is being written by five 15 year olds who just wrote "Hello World" last week for the first time, and "Project B" is being written by twenty crusty old timers, and "Project A" has a three person "community" and "Project B" has zero community inspecting the source, I still guarantee that "Project B" will blow "Project A" out of the water. Open source, closed source, it really does not matter.
Another thing that I find fallacious about this argument is the continual assumption that "open source" means "free." The two are not mutual ideas, not by a long shot. Nor are they mutually exclusive. Historically speaking, UNIX is "open source," but hardly "free." Indeed, the original BSD386 project was due to the desire for there to be a free UNIX. One of the reasons why so much System V source code has ended up in various UNIXs over the years is precisely because Sytem V was open source, and SCOs lawsuits exist because System V was not free! On the other hand, there is plenty of free software that is not open source. Just got to any shareware repository and find a piece of freeware that is pre-compiled and that does not include source code.
What really matters is who is writing and reviewing the code, and money tends to attract the continued writing and review of code much better than whatever it is that actually motivates FOSS coders. Sure, some FOSS projects (Apache, Linux, BSD, MySQL, etc.) attract top talent, but just taking a look at Source Forge shows that the vast majority of Open Source projects go nowhere. To decide that FOSS is the best possible method of development and quality control based on Windows vs. Linux or Oracle vs. MySQL or IIS vs. Apache or PHP vs. Java, or whatever is silly. That is like saying that "a Dodge will always be faster than a Chevy" based upon a comparison between the Viper and the Corvette.
One of the reasons why these projects are able to attract such a large pool of developers and testers has less to do with the fact that they are "open source," but the fact that they are free. Only a minute percentage of Linux users ever touch their source code, let alone look at it (or even care about it). They are attracted by its phenomenal price/performance ratio. The same can be said for any FOSS project. Thanks to the widespread usage of various package managers, it is fairly uncommon for most mainstream Linux users to even compile from source, let alone modify compiler flags or make changes to source code. If these packages were closed source but still free, most of their users would still use them and be testing them.
The vast majority of lines of code are written under the radar of most people and do not get any attention. Try comparing a small sample of each type of software. Take a few dozen random items from Source Forge that are in at least a "release quality" state and compare them to a few dozen freeware applications. Then evaluate the difference between closed source and open source. I really cannot tell you what the results will be, but I do know that many of the open source pieces of code that I have used, outside of the "big stuff" (various UNIXs, Apache, MySQL, PostgreSQL) are not so great. For a project that lacks glamour, it is hard to attract someone to spend a large amount of time seriously working on it. It is that simple.
Mind you, I am not against open source or free software whatsoever! I use it all of the time in my day-to-day life, especially FreeBSD, Apache, MySQL, and Perl. But I also use a lot of paid and/or closed source software as well, like Windows, IIS, Oracle, Microsoft Office and so on. Some of my favorite pieces of software are simple shareware applications: Notetab Pro and ThumbsPlus immediately come to mind. It is very rare that I have ever wanted or needed to "look under the hood" of a piece of software. Tomcat/Jakarta required me to do so to find out why it was not behaving as documented. Indeed, most of the time that I have had to look at source code, it was to compensate for poor documentation, not to actually make any change or satisfy a curious itch. I was grateful to be able to inspect the source code, but I would have preferred better documentation instead.
Many of the arguments that I hear in favor of open source compare Windows to Linux, or Internet Explorer to Firefox. Windows vs. Linux just shows that the Linux coders are better, smarter, and more well organized than Microsoft's Windows coders. Microsoft has had something like ten years to get Internet Explorer right, and they still have not managed to get it nailed down. This is not news. The fact is, closed source shops consistently crank out many products better than Microsofts too. One can just as easily compare OS/2 Warp or BeOS or MacOSX to the version of Windows from the relevant timeframe, and Windows still falls short on many (if not most) benchmarks like security, stability, usability, and so on. I note that it is rare for someone to compare MySQL or PostgreSQL to Oracle or Microsoft SQL Server in an "open source versus closed source" debate. Why do we never hear ".Net versus Mono?" Even comparing Apache to IIS is difficult, because IIS is a significantly more ambitious piece of software than Apache.
Open source, in and of itself does not produce better code. Better coders and better testing produce better code. It is that simple. When a closed source shop has the better coders and better testing, they write the better software. When an open source project has better coders and better testing, they write better software. To think that just because a piece of code can be modified or inspected by anyone and everyone means that the best coders and testers will be modifying and testing that code is just not correct.