CXO

Vendor advice: How to separate the signal from the noise

Tech vendors tend to use terms like "best practices" pretty loosely when promoting their solutions. Here's how to separate what's real from what's marketing fluff.

You've probably participated in the proposal dance with a software or services vendor. After pleasantries are exchanged and a few dozen PowerPoint slides presented, you get to the "best practices" and "quals" portion of the show. Impressive-sounding metrics and results are shown, and big-name companies that used the technology or service flashed on the screen, all implying that merely by purchasing the product on offer and following some "best practices," your company, too, can be fitter, happier, and financially secure.

What is a best practice anyway?

Like most overwrought concepts, the idea of best practices is grounded in a simple and largely true idea: that there's a right and wrong way to perform a task if you want a certain result. The best practice to avoid a speeding ticket is to stay under the speed limit, and the best practices to lead a healthier life are diet and exercise. However, in the case of technology and services, purveyors of best practices tend to ignore a second truism that should be familiar to anyone who ever set foot inside a statistics class: correlation does not imply causation.

Special Feature

Special report: How to choose and manage great tech partners (free PDF)

In a world increasingly dominated by cloud computing and outsourcing, vendor management has become a core competency of running a great IT department. This ebook, based on the latest ZDNet/TechRepublic special feature, looks at how you can maximize those partnerships.

That marquee company that installed a new technology platform and saw a 15% ROI in just nine months may have concurrently launched a massive cost cutting program. The retailer with a 5% sales increase after launching their mobile app may have been in an industry where peers had a 15% sales increase over the same time period due to general market conditions. And of course, there's the famous example from every introductory statistics class: Ice cream sales rise in near perfect correlation to shark attacks every summer. Do ice cream sales cause shark attacks?

Cerebrally, we understand that correlated events don't automatically have a causal relationship, yet IT leaders will often sit in rapt attention as a vendor suggests the mere presence of their technology or process will drive success at their company.

My best practice might not be yours

The other fundamental flaw with best practices is that they ignore your organization's current capabilities and abilities and the relevance of the practice to the matter at hand. Most IT leaders have heard of Agile software development, and after years of hype probably have a sense of what types of organizations and specific efforts for which Agile is appropriate. Take a look at one of the newer methods being touted as a best practice, like DevOps, for instance, and overzealous talking heads are pitching it as the solution to every problem.

Separating the signal from the noise

As you consider services or technology tools, try to identify how the tool or service directly influenced the result. When you hear a vendor claim that doesn't seem to pass the sniff test, ask how similar industry peers that did not implement the same tool or process fared in an attempt to determine how the practice under consideration contributed to the result versus an unrelated market or organizational condition. Ask what other efforts were undertaken at the same time. If some software was a small piece of a massive organizational restructuring, the results are likely more readily attributable to the reorganization than the software.

Even if a best practice seems legitimate, assess your organization versus other organizations that used the practice successfully. For example, hiring a chief digital officer may have accelerated digital at your competitor, but would be an expensive failure at an organization that sees little value for technology beyond back office computing. Similarly, if your organization suffers from disorganization and poorly-planned processes, simply running the same software as more well-organized businesses will only serve to automate your organization's own poorly managed processes to little real benefit.

As with any practice, there are lessons to learn from peers and other companies, and it is important for every successful business to scan and monitor their industry and the market at large for emerging trends and practices that are relevant. However, as you identify and consider new practices, be they technology, processes, or organizational models, realize that best practices are not magical, and not always adaptable or relevant to every organization. Approach each best practice with healthy skepticism, attempt to determine how the practice contributed directly to the suggested result, and test its relevance to your individual organization or team. That's a best practice that's always going to be effective.

Also see:
istock-682380922.jpg
Image: iStock/jacoblund

About Patrick Gray

Patrick Gray works for a global Fortune 500 consulting and IT services company and is the author of Breakthrough IT: Supercharging Organizational Value through Technology as well as the companion e-book The Breakthrough CIO's Companion. He has spent ...

Editor's Picks

Free Newsletters, In your Inbox