Most CIOs I talk to have sworn off software testing programs, opting instead for what I call the Service Pack 1 release cycle, which means no software goes on their production systems until the first set of patches is released. But this hard line stance on the use and deployment of prerelease software may cost your company more than you realize. Given the soft economy and most software companies’ desire to continue producing and selling software, corporate technology buyers are in a great position to negotiate with vendors for favorable treatment during a prerelease testing phase.
First, let’s be clear on my definition of beta software. I'm not talking about access to code you can download from a vendor’s support site. Of course, you should recognize that most companies have employees who are already doing this kind of beta testing—sometimes it’s on their own time and own machines, but many times it involves using company time and assets.
Rather than relying on this limited software testing, the beta process to which I’m referring involves real partnerships between companies and their software manufacturers. As beta testing is happening at some level anyway, you must determine the best way to harness the knowledge generated from these activities and leverage that activity to produce better products and services while building the relationship with your software vendors.
Of course, you may ask, "Why should I risk my job on the company’s adoption of prerelease software?" First, beta testing allows your company to get early access to new technology. But you should only consider this strategy if it allows you to optimize existing processes or enables the implementation of new processes that you couldn’t use before because of immature technology or the price tag associated with the implementation.
Even if you choose not to widely deploy the technology, access to the beta process provides a better understanding of industry trends and technologies by giving you an in-depth understanding of new technology and the ability to talk to the developers and designers who implemented it. You get a deeper understanding by getting a chance to learn the “why” and “how” rather than just getting the “what” information after the software is released.
Another important benefit is the direct technical support from manufacturer during a prerelease cycle. Software companies recognize the investment enterprises make in testing the software and are generally willing to reward them with technical support commensurate with the amount of feedback you give them. Many times this includes direct access to vendor software developers and architects.
Once the product is released, the software company has neither the incentive nor the time to give their customers access to this level of support. The best you can hope for is a strong support relationship. If you’re using their product in a unique or marketworthy way, you may even be allowed to participate in press releases, case studies, or other marketing activities. And if your company would benefit from the attention given to those early technology adopters, this type of free publicity would be worth pursuing.
Why not beta?
Of course, some companies might choose not to participate in beta testing because no matter how you slice it, a decision to be a beta tester involves investing company R&D dollars to test someone else's product. And given that most companies barely scratch the surface of the software investment they’ve already made, it’s likely the current investment in technology may be sufficient and more cost effective to implement.
Companies that choose to beta test must realize that dealing with sparse documentation and spotty support after major release milestones could double or triple the cost of developing or deploying internal systems or software based on the beta technology.
Just remember that a decision to include prerelease software in internal systems or software initiatives should have a clear benefit to your company's bottom line.
Using beta software effectively
Once you’ve made the decision to implement a system on prerelease technology, you'll still want to minimize your risk. Here are some basic do's and don’ts to consider:
- Don't roll out any unreleased products—or any internal products based on unreleased technology—unless the manufacturer has a vested interest in your success. This means that they’re either providing you with advanced technical support or, even better, they’re committing to a major advertising campaign based on your solution. Once you’re linked with the company in the public eye, the software company can’t afford to let you fail.
- Do isolate new technology into its own physical or virtual systems during early testing. I worked with one customer whose beta product generated so much unexpected network traffic that it crippled production systems. To minimize potential exposure, consider using either dedicated physical machines or products like VMware that allow you to create virtual test environments without exposing them to production environments.
- Do expect and require a substantial benefit from the manufacturer for investment in their testing cycle. Unless you can quantify the benefit, you should make it clear to the manufacturer—and to your own technical employees—that you’re better off waiting until the manufacturer releases the product to the public.