Recently I facilitated a workshop for employees from a dozen nonprofit agencies who wanted to learn how to evaluate software vendors and their programs. On the agenda was a CPA who led a hands-on tutorial demonstration of a popular commercial fund-management application for nonprofits.

The group included hallway gurus, bookkeepers, and chief financial officers from agencies of various sizes. They all had a couple of things in common. First, they wore the IT department hat in addition to their other duties. Second, they were all charged with choosing and implementing software solutions for their respective organizations.

Ask everyone the same questions
This group wasn’t sure what to ask when someone tried to sell them a custom or off-the-shelf application. I recommended a tried-and-true method: Create a checklist of key questions to ask each and every vendor, and keep the answers in electronic format, whether in a word processing document, a spreadsheet, or a database.

Keeping a formal record of vendor responses to questions comes in handy when you want to compare vendors and their applications on a side-by-side basis. This week, I’ll share some of the questions that came up when the workshop group discussed the need to buy a program that can be supported by companies that don’t have a full-time IT person on staff.

I invite your help in creating a list of standard questions that’s as comprehensive as possible. Just post a comment if you want to put in your two cents about evaluating products for supportability.

Evaluating the vendor
In large companies, you don’t even get to think about buying software from the local software wunderkinder who formed a company as part of their undergraduate studies just because they’ve written a great program.

To become vendors of big companies, software vendors and consultants usually have to pass rigorous tests by the accounting department to establish creditability. Here are some of the questions your sales rep should be able to answer:

  • How long has the company been around?
  • Will the vendor be around for the long term?
  • Does the vendor have a sizeable client base?

It’s okay to ask for references. If you ask for three and the vendor gives you a two-page list, that’s a good sign. If the vendor can’t even provide three, you may want to consider that a red flag.

Report options
Before you invest a ton of money in software, hardware, and training for a system, you must ask yourself one question: What output do I expect from this system?

To answer that question, start by making a list of the titles of the reports you know you’re going to need to run every day, every week, every month, every quarter, and every end-of-year.

Then, for every report title you identified, list the columns or fields you want to see on that report. Finally, ask yourself these questions:

  • Assuming I configure the tables and enter the data correctly, does the application have built-in report templates for all of the reports I need?
  • If the application doesn’t have built-in templates for some of the reports I need, is there a utility that lets me customize my own reports?
  • If there is a custom report-builder, how easy is it to use?

Compatibility with existing systems
Don’t fall in love with an application’s front end and then find out it won’t run on your current system. Start with these questions:

  • What are the minimum hardware requirements for installing and running the program on the server? On workstations? Specifically, how much available space is required on the disk drive, and how much RAM do I need for optimum performance?
  • What are the minimum operating system requirements for the server and the workstations?
  • Does my current infrastructure meet those minimum requirements?

I know it’s an old-fashioned concept for most IT pros, but end users learn how to use software as much with lessons they can read as they do with the try-and-see-what-happens method. Before you buy, ask these questions:

  • When I purchase the software, do I receive a printed manual at an extra charge?
  • Or am I expected to use the online help feature to look up how to use the application and buy the print- or CD-based materials separately?
  • Are explanations of error messages included in the documentation?
  • Are the directions clear as to how someone uses the software effectively?
  • Are directions enhanced by useful examples where appropriate?

Call me old-fashioned, but I like to get at least a CD when I buy software. In writing disaster-recovery plans for an important financial application, I had to report an application for which my client had no original or backup copies of the software. In the world of business continuity planning, that’s, of course, a no-no. Everything about your rights is in the agreement, but ask your representative:

  • When I buy the application, do I get original copies of the software on CD or disk?
  • Am I allowed to make backup copies of those disks?
  • Can I upgrade or patch my software online?

Online help
I throw this one in on behalf of all those power users out there who don’t have anyone they can call for tech support, and so they’ve learned to bail themselves out of tough situations by drilling down into online help. When you’re testing the software, be sure to answer these questions:

  • Is context-sensitive help available by pressing [F1]?
  • Are mouse-over tips available for toolbar tools, buttons, and other program icons?
  • Is the interface user-friendly enough to be used with a minimum of computer expertise?
  • Is there a Web site I can visit for troubleshooting tips, lessons, or tech support?
  • Does the software accurately evaluate and validate user input?
  • When an entry is invalid, is it easy to redo the entry?

Some applications appear to process data and repaint screens with the speed of lightning—on the demo running on the sales rep’s laptop. Be sure your prepurchase testing includes determining the answers to these questions:

  • How long does it take to boot up when I launch the program?
  • Are there any unacceptable delays in executing tasks?
  • Does the software pause without explanation?

Import and export options
In every shop where I have provided tech support, end users have requested the ability to capture or export data from a proprietary application so they can analyze it or use it to generate a custom report. Though the location of the functionality may vary from interface to interface, self-respecting databases ought to provide favorable answers to these questions:

  • Will the program allow me to export data to an external file type, such as a database, spreadsheet, or plain text (e.g., delimited and fixed-length), so I can manipulate that data for special reports?
  • Will the program allow me to import data from external sources, such as databases, spreadsheets, or plain text (e.g., delimited and fixed-length), so I don’t have to key my old data from scratch?

Training, support, and updates
Here’s a big surprise: Software that is complex enough to meet the needs of a business often is complex enough to require end users to be formally trained to use that software efficiently. So don’t be surprised when you discover that training, support, and upgrade costs may be as much as 25 to 50 percent of the cost of the software itself. You never know what options are available until you ask, though.

  • Is any free training for my staff included in the purchase price of the software?
  • Will you train my staff on site, or do I have to send them out for training?
  • What kind of tech support do I receive for the purchase price of the software?
  • Is there a toll-free number to reach tech support?
  • What are the hours of operation of tech support?
  • How many free calls can I make per week or per month?
  • How responsive is the customer service?
  • Are there any for-fee support options available, and what are the costs?
  • Are there hidden charges?
  • Is the software updated regularly and when?
  • Are updates automatically snail-mailed or e-mailed?
  • Will my sales/support representative inform me when minor patches or major upgrades are available?

Backup and disaster recovery options
Sometimes an application is too nice. It doesn’t remind us when no one has run the backup utility for six months. Ask your vendor about what it can do for you in case of interruption of computer services.

  • Does the software make it easy to perform manual backups? Schedule automatic backups?
  • Does the company offer any free or for-fee disaster recovery support, in case I have a catastrophic hardware failure?
  • How soon can the vendor provide on-site help or replacement equipment?

Only the beginning
If you’re looking for a set of formal criteria by which to evaluate the supportability of a commercial application, I hope this checklist helps you get started. The most important thing to remember is that, when you’re dealing with someone who wants you to write them a check in exchange for a product or service, it’s okay to ask a lot of questions. Good vendors won’t mind answering.

Add your tips to the prepurchase checklist

Do you have a formula for evaluating software? To share your experiences with fellow TechRepublic members, post a comment or write to Jeff.