By Tim Landgrave

This article originally appeared on our sister site, TechRepublic.

As the person responsible for making major technology investment decisions, you’re likely to make at least one big decision each year. The definition of “big” is different based on company size. It may be a project that costs $1 million for some companies and $50 million for others. Or it may be a project without high costs, but one whose impact will be felt by the company for years (for example, changing the operating system on every desktop in the organization or making a decision to outsource a major portion of your business). For these scenarios, most companies have a well-defined process that includes a formal bidding and proposal process, a selection committee, and an official “award” of the business. These projects tend to be won by large vendors with significant consulting and implementation teams on whom a CIO is willing to take a career risk.

Interestingly, because there are so many people involved in this decision, the failure of a large project is rarely the cause of a CIO’s dismissal. Rather, it’s the failure of a smaller project whose financial impact was grossly underestimated that the CIO has the most exposure. These projects typically fall to secondary technology vendors either because their larger vendors won’t do a smaller deal or because the CIO is willing to take a chance in order to save money. These projects also typically involve specific domain expertise that secondary vendors have and their larger counterparts do not. Given that these decisions aren’t perceived to be large enough financially or broad enough to justify the committee approach, what should a CIO use for the criteria to select the proper vendor? Focus on the big three: technology, people, and process.

Given that the systems being evaluated are typically for departments or for business units that have a very specific vertical need, you might be tempted not to insist that the technology used for the solution be on your corporate standards list. But even though you may not be able to dictate the exact technology, you still have a responsibility to the company to make sure that the platform and the language chosen will be around for the life of the system. You also need to be confident that there will be support available outside of the initial vendor in case it ceases operations.

You also have to be very firm with these vendors when demanding access to their source code. While you may not be able to ask an Oracle or Microsoft to give you source code access, you also don’t have to go to bed at night worrying about whether they’ll be around in the morning to support their applications. You’re not being unreasonable to demand that a smaller company sell you an application license that includes the source code (for internal use only) or to demand that a mutual agreement with a company that escrows source code makes it available to you under specific financial conditions, such as bankruptcy, acquisition, or dissolution.

The mistake that many CIOs make in these negotiations is that they get hung up on the technology decision and forget that the success of the system will ultimately come down to the people who developed the system and the ones who have to implement it. In my experience, the single most important “people factor” in these decisions is the domain expertise of the developers and implementers. For example, if you’re purchasing a logistics system for your distribution group, it’s critical that the consultants who will develop (or have developed) your solution have specific experience with distribution issues in your industry. You’re well within your rights as the consumer to request resumes or interviews with the people who will be directly involved in your implementation.

You should also ask for customer references in your industry. If they have no customer references in your industry, you should ask for customer references in similar industries. When I work as a business consultant to help CIOs make these decisions, I’ll typically ask a vendor for the name of a reference who had a major problem during implementation but is still a customer. I like to understand how the vendor deals with major setbacks because you learn much more about the vendor from how it responds to failure than how it deals with success.

I’ve seen companies that have well-written products using state-of-the-art technology with top-notch developers and consultants who can’t seem to get any system completed or installed on time or under budget. If the vendors have such great tools and stellar people to use them, then what else are they missing? It’s most likely proven methodology for development and deployment. When evaluating these secondary technology vendors, you should ask the sales rep and one of the help desk or support engineers to walk you through their development and deployment methodology. Why? Because if the sales person (at the beginning of the process) and the support person (at the end of the process) can explain their methodology, then there’s a good chance that it’s ingrained in the organization and not just a spiel that the lead technology person gives to convince you that the methodology is widely used in the organization.

Sum it up for success
If you’re evaluating a single vendor, make sure the technology, people, and process are all acceptable before you engage the company. If you’re evaluating two or more vendors, use these three areas as your basis for comparison. Even though these decisions may be for secondary technology initiatives, this selection is likely to have a much greater impact on your job evaluation because you’ll likely be making the decision on your own. Having a set of guidelines will help you make a more informed decision.