Don't lose customers by overpromising and underdelivering because of weak predictive analytics. Here are tips on benchmarking and improving your predictive capability.
From the carnival fortune-tellers to the advertising gurus on Madison Avenue, the practitioners of predictive magic have always wielded a powerful influence over those who have a vested interest in knowing what the future holds. This influence is intertwined with skepticism and risk, which can be an estimable challenge if predictive analytics is the lynchpin in your next big strategy. When incorporating predictive analytics into your product offering, make sure you align your predictive capability with customers' expectations.
Benchmarking predictive capability
The fastest way to go out of business is to tell your customers you can do something when you really can't. Tech companies are notorious for selling the idea that their product can do wonders, and then customers discover it has bugs. To be fair, consumers, especially early adopters, are generally aware that new technology doesn't always work the way it should -- but that doesn't pardon Sales and Marketing from misleading potential customers if they know a product has issues.
This situation is exacerbated with products that incorporate predictive analytics because there's no way to know the future. However, we can be at least somewhat confident about how well we can predict the future.
If your product offering incorporates predictive analytics, a fundamental set of metrics that you should know is your predictability and the value your model has over someone throwing darts. For instance, if a horserace handicapper touts his ability to pick a winner 30% of the time, that doesn't impress me, because I know the favorite wins the race about a third of the time. If necessary, collect customers' feedback about how well you predicted their behaviors or desires, and track this on a control chart. This will put you in a good position to improve your predictive capability if that's part of your strategy.
Improving predictive capability
If your goal is to create a product or service that's merely competitive (i.e., it places you in the game), average predictability may be enough. If your strategy depends on the product being distinctive or a breakthrough, settling for average predictability won't do.
After benchmarking your current capability, the most important thing to determine is where your predictability needs to be to properly support your strategy. For instance, you may know from your customer feedback that your recommendation engine is hitting around 60%. You also know through industry benchmarking that 60% is about average, 80% is impressive, and anything above 90% is remarkable. It may be tempting to shoot for 95%, but there may be huge costs and risks involved. It might make more sense to shoot for 80% and take distinctive positioning in the marketplace at less risk and cost. Whatever you decide, make sure it's clearly articulated to your data science team, and then take a structured approach to get where you need to be.
Managing customers' expectations
Improving predictive capability isn't the only way to put yourself in alignment with customer expectations -- you can also manage customers' expectations. There's nothing inherently wrong with lowering customer expectations, as long as it meets your strategic needs. It's much better than giving off the false impression that your predictive capability is higher than it is. The marketing may be tricky, but there are honest and ethical ways to position yourself with your market, even though your powers of prediction may come up short once in a while.
One way to do this is to make your unpredictability part of the fun in your offering. I recently spoke with the CEO of Graze, an innovative company that's using predictive analytics to disrupt the snack industry. I was a bit startled when he mentioned that his customers don't get to choose what snacks are delivered to them -- Graze just sends them what it thinks they will enjoy. I challenged him on this idea, thinking it was too assumptive and risky, but he corrected my perception. He explained that his customers enjoy not knowing what's coming in the mail -- they want to be surprised by what Graze designs for their next snack.
Graze has done a terrific job of managing expectations. Customers are not counting on Graze's predictions to be accurate, so they aren't terribly disappointed when they're not. Instead, customers are in on the game and are pleasantly surprised when Graze gets it right.
Meeting customers' expectations with technology is difficult, but meeting them with predictive analytics takes this difficulty to a whole new level. You want to be innovative and cutting-edge, but you don't want to lose customers by overpromising and underdelivering on your predictions.
Learn to meet customers' expectations by first learning about your predictive capability. From there, you should use the two levers of improving predictive capability and adjusting customers' expectations to make sure you're delivering on your promise.
Fortune-tellers are entertaining, but you don't want to be the carnival act of your industry.