CXO

Be wary of vendor references, but don't discount their value

It's unlikely that a vendor will refer you to a customer that has anything but favorable things to say about its products or services. If you're helping clients evaluate product offerings, make the most of vendor-recommended references.


If you're considering purchasing a software or hardware product from a vendor, you're going to perform an evaluation. Of course, the amount of time you spend on due diligence is directly related to the investment you're making in the product.

I recently helped purchase a simple scheduling package that cost my business $2,000. We didn't perform a lot of diligence other than to make sure the package would solve our business need. A few years ago we purchased a Customer Relationship Management package for more than $100,000. The level of up-front evaluation and diligence for that purchase was much higher.

One of the activities that should be on your evaluation checklist is talking to companies that use the product. The purpose of checking references is to get past the marketing and sales hype and talk to some honest-to-goodness customers. The theory is that these companies will give you a more honest picture of how the product and the vendor actually perform in the real world.

How much faith do you place in a reference?
Everyone is aware of the potential pitfalls associated with references. First, when you get ready to check references, the vendor is going to give you the names of customers it knows will give a good reference. There’s nothing wrong with that. Every vendor has trophy accounts and accounts that have experienced problems. If you were the vendor, you wouldn't provide reference accounts from customers who had bad experiences.

No one is claiming that the reference customer would lie to you. I expect that any references will give you an honest opinion and honest perspective of the product and their relationship with the vendor. It’s just that these references will always have a positive story to tell.

As a prospective customer, the first question you need to ask about references is whether you'll accept the ones that come from the vendor, or whether you'll try to get some unsolicited references. If you want to get unsolicited references, you have to know something about the current customer base. Sometimes the vendor will include the names or logos of licensed companies on its Web site or in marketing brochures. One approach is to pick out companies that you know are customers and call them.

Unsolicited customer references
Using unsolicited customer references may put you in touch with someone who has had less than successful results, giving you an opportunity to see how the vendor has responded. It's also possible that the customer will be happy, in which case you don’t get much more information than you would from an official reference account.

The disadvantage is that you have to do some research, which takes a lot more time. You have to find prior accounts on your own. You may have to cold-call that company and look for the right person, and you have to hope they will talk to you.

Talking to official references
If you're going to talk to “official” company references, there are a number of things you can do that will help you get the most out of the discussion:
  • Chances are, the vendor will have a number of potential references—pick three or four. One person can call each company or you can split the calls up among several people. If you split the reference checks, make sure you get the people together first so that everyone follows a similar process in the reference call.
  • Have a script or cheat sheet to help guide the reference discussion. You don’t want to have many great ideas for the discussion and then forget your questions. If you have multiple people making the calls, this also ensures that all of the reference checkers are asking a similar set of questions.
  • Try to get companies that are as similar to you as possible. This could mean different things depending on the product. For example, this may mean that you want to find companies in the same industry as you. It might mean that you want companies that are the same size as you. It could mean that you want companies that process as many transactions as you. So, depending on the type of product, look for companies that are similar to you in the ways that matter.

Once you've identified the references, take advantage of the time you spend with them. I've had prospective customers call me for a reference. Some are very well prepared. Some potential customers are very vague and don’t know what questions to ask. Here is a guide that will help you in the reference call:
  • Conflict of interest. Ask customers if they or their company receives any reimbursement or credit from the vendor for providing a reference. Normally this does not happen, since it creates an obvious conflict of interest. However, if the reference customer is getting a reduced price on maintenance, or anything else, you should seriously question the validity of the reference feedback.
  • General satisfaction. Ask a few general questions such as, “Are you happy with the product?” to get you started, but then drill down quickly. Ask, “What about it satisfies the needs of your company?”, “How do you use the product?”, and “Has it met all of your expectations?”
  • Business value. Most products claim to have a high return on investment (ROI). Find out what the reference customer says. Ask, “Have you ever done an ROI after implementation?”, “What was your prior process and how does this product make you more effective or efficient?”, and “What type of value does the product provide to you?”
  • Implementation. Ask specific questions about implementation. Don’t just ask “How hard was the package to implement?” Ask questions like, “How long did it take you to implement?”, “What were your biggest implementation challenges?”, “What type of training was needed?”, “How easy was the product to learn to use?”, and “How did the vendor help during implementation?”
  • Support. Ongoing support is where many vendors have a competitive edge. Again, don’t ask “How is the support?” Ask more specific questions, such as “How many support calls have you placed to the vendor?”, “What is the typical turnaround time to resolve your problems?”, “How knowledgeable is the support team?”, and “Has the product ever failed or gone down?”

If it is practical for the product you're purchasing, you might also ask if the reference customers would be willing to have you come to their facility to see how they're using the product on an ongoing basis. This type of request may need to be made through the vendor.

No large purchase should be made without talking to references. This is part of the due diligence process. You know ahead of time that the references will be positive, but you can still learn a lot about the product by delving deeper into the customer’s experience. During the discussion you want to stay away from “How do you feel …?” questions and focus instead on the facts regarding the customer’s experiences. This includes product implementation and support, and the business value that the product has provided. If you do this, you can gain a great deal of insight into what might happen to your company if you purchase the product. That is ultimately the purpose of the reference check.

Editor's Picks