Well, a number of things. But RedHat HP are up there on my current "things I am enjoying least right now" list. I suspect other distro's are providing a similar experience to other Systems Engineers.
My "real world" is the high end server world. We have a solution we manage for a particular customer which is mult- tiered. Front end authentication and access to the solution has been built by one vendor who chose to go with Sun hardware, Solaris and Oracle for the database. All wise enough choices. This is the end of the solution that sees very little trouble.
For some odd reason or other (probably political or penny-pinching) one of the backend sections - which sees a heck of a lot of database activity and file storage - was originally installed on a number of RedHat 7.3 servers using recycled Intel hardware from a previous project. Now, suddenly the user base has grown and the backend solution needs to be upgraded. So we moved to bigger hardware. This is where it got - um - interesting.
We installed an 8-way intel server - upgrading from a 4-way box, and the the database got slower. It's PostgreSQL, and the 8 cpu support is somewhat recent. In fact we couldn't find anyone else using POstgreSQL on an 8 way box in a production environment. It seems we are now at the "bleeding edge" -- oops!
So in their infinite wisdom, the project managers (who consult with, but never actually listen to, the techies) decide to purchase a 4-way dual core 64-bit Athlon based machine. Introducing not one, not two but THREE new technologies in one Swell Foop!
Now we can't access one of the RAID arrays, because RedHat don't support that driver on 64-bit kernels, and the manufacturer doesn't either. We can't import the data directlty from the old datbase to the new 64-bit database, so we have to compile a 32-bit version (on a 64-bit kernel) as in intermediate step to migrate it. More storage pleeeeez! And finally -- the database is STILL SLOW anyway, so who cares?
The whole process costs a ridiculous amount of money and time, and the users are ready to crucify the customer.
The experience has told me that you will save money if you do it right the first time. But I knew that already. Also told me you really need to understand the technology when you spec your hardware. But I knew that already, too. On the Linux side, though, the message is a little more frustrating.
Vendor support has been disappointing. When you find a solution crippled because you *have* to move up to new operating system (EL3 or 4) for new server hardware, but some of your peripherals (SAN cards, in particular) and some of the software you've been using won't scale to the new environment you have to start questioning wether open source software was a wise choice.
If it was Oracle rather than PostgreSQL, for instance, we could get vendor support - even to the extent of recoding or patching the database to make it work. I have seen Oracle, as one example, actually do this on previous projects. You can't get that with PostgreSQL. If you haven't got the skills in-house to contribute to coding an opensource project, you have to think very carefully about where you implement it, how you are going to support and maintain it, and what risks you are exposing yourself to where it is installed.
Now, let's be fair. If we had implemented this on Windows... well - you could forget getting past square one. It just would not have flown. The problem of scalability involved hardware, not just software. IO throughout, memory bandwith, FSB speed, and the whole hardware architecture all needed to be considered, but the people spec'ing the project hardware did not understand these concepts let alone how to quantify the needs of the application. We needed SPARC systems when we bought Intel/AMD, and that was and is the heart of the problem.
Linux, right now is positioned very snugly between the midrange Unix market and the Wintel space. It does well there. But at the higher end of the Intel/AMD server market, it has teething issues. It's moving into new territory, and developers haven't caught up with the hardware yet. It's expensive hardware, too, so unless hardware vendor s like HP start getting involved (Hmm. HP-LinX?) chances are open source developers are not going to able to afford to catch up in the short term. It's fine if you are going with a brand new solution, as long as you understand what you are doing and why. But you should think carefully about migrating older solutions. You may not be able to do it as smoothly as you would expect.
