You've hit on one of the marketing issues that large sever manufacturers like to talk about in their advertising. There are some advantages and some disadvantages to splitting server functions between many machines or consolidating many functions on a few servers.
Consolidating many functions on a few machines has the following advantages:
- Less man power is required to administer fewer machines. There are a set of tasks that will have to be performed on each machine. These include network configuration, installation of patches, maintenance of user accounts, file system security configuration, and other stuff. If you have many machines then these tasks will have to be repeated for each machine. If you have one machine then you only have to perform these tasks once.
- Lower cost for hardware maintenance contracts. If you have one machine then you will only have to purchase hardware maintenance for that one machine. If you have many machines then you will have to purchase hardware maintenance support for each machine.
- Lower probability of failure. Just like in RAID disks, the probability of failure on servers rises each time that you add another machine.
On the other hand there are distinct advantages to distributing functions across many machines. Some of these are:
- Less impact when a server fails. Hardware will fail. Period. If a server performing many functions fails then you lose all of those functions until that one machine is repaired. If a machine performing a single function fails then you only lose that one function.
- You can configure/tune each server to maximize its ability to perform one function at the expense of unused resources. Different server functions use different computer resources. Trying to consolidate many functions on one machine means that you may have to compromise server tuning to try to accommodate different diverse resource usage requirements. For example, a disk server benefits from having a lot of RAM devoted to disk cache, which is in kernel space. A database server benefits from having a lot of RAM for application data storage, which is in userland. The two requirements are mutually exclusive. If these two functions are on separate hardware then you can tune each server to maximise its performance for its function.
- Many small machines can be purchased separately over a long period of time, thus spreading out the impact on corporate budgets.
These are just some of the arguments in favor of each position. Since you also seem to be saying that each server is overbuilt for its role I would say that excess hardware capacity means that you can keep these machines for a long time. In other words you won't have to upgrade your hardware any time soon. While this is a questionable strategy due to the dramatic drop in price of most hardware over time it may be a smart political move. The IT department will have longer periods of time between making budget requests for new equipment.
In the end it's a judgement call.
