Innovation

AI missed repeat in 2017 Kentucky Derby, but outperformed the experts

AI-based swarm intelligence surprised the world by forecasting the top four finishers in last year's Kentucky Derby. It didn't fare as well in Derby 143. Still, it outperformed the handicappers.

When an artificial intelligence-powered "swarm" correctly predicted the order of the top four finishers in last year's Kentucky Derby, the surprising story got picked up around the world.

For the 2017 Derby, artificial intelligence learned the same lesson that has been beating up dyed-in-the-wool handicappers for over a century: horse racing is the most unpredictable sport on the planet, and the Kentucky Derby is the most unpredictable race in the sport. That's what makes it so exciting.

This year's swarm only predicted two of the four top finishers—Always Dreaming and Classic Empire—and it had the pair in the wrong order. The swarm picked Always Dreaming fourth while the colt ended up controlling the race and winning easily. Classic Empire was the disputed top selection of the swarm but he barely nabbed fourth place in the race on Saturday.

The swarm's picks for second and third—McCracken and Irish War Cry—finished eighth and tenth, respectively.

The swarm was thrown into disarray by the unlikely performance of two longshots: Looking at Lee (second place) and Battle of Midway (third place). Both horses unexpectedly ran huge races that left racing experts scratching their heads.

SEE: How 'artificial swarm intelligence' uses people to make better predictions than experts

In fairness to the AI swarm, there were a couple key factors that contributed to the unpredictable outcome of the race.

The first was the weather. It's rained for three straight days in Louisville leading up to the Derby and that made the track wet and sloppy. Since all of these horses are three year olds, most of them had rarely, if ever, run on a wet surface. Whenever it rains for the Derby, it always throws the race into a much more unpredictable state because you're not sure how the track will play or how a lot of these horses will react to it.

The other factor was that since the rain started on Thursday, the track has been favoring front-running speed horses and horses that stay down near the rail. As a result, in the past 72 hours the public moved frontrunners Always Dreaming and Irish War Cry ahead of Classic Empire, a stalker, and McCracken, a come-from-behinder. The public bet down Always Dreaming and Irish War Cry to become the 9-2 co-favorites and bet up McCracken to 6-1 and Classic Empire to 7-1. In other words, a lot more more was placed on the first two because of the rain and the fact that the track favored speed horses on Kentucky Oaks Day on Friday.

The swarm didn't have that information when it made its picks on Wednesday. It's likely that if the swarm had been executed on the morning of the race, the results would have been different. It likely would have favored the speed horses, just as the public bettors did.

Still, it's unlikely that the swarm would have moved Looking at Lee or Battle of Midway into its prediction for the top four finishers. But, it might have also changed the longshot picks that the swarm targeted. We should note that among the four longshots the swarm recommended keeping an eye on, Battle of Midway was one of them.

It's also important to remember that when the swarm predicted the 2016 Derby, it chose the four horses that also ended up being the top four betting choices. The fact that the top four horses finished in the top four is actually pretty rare in the Derby. Because of how unpredictable the race is—with the horses being so young and the field being so large at 20 horses (most horse races are capped at 14).

SEE: How to Implement AI and machine learning (a ZDNet/TechRepublic special report)

The bigger questions, of course, are whether swarm AI outperformed polls and experts—as it's done in the past—and if it's still a useful tool. Since it hit two of the four places and Looking at Lee and Battle of Midway were barely part of any conversations about potential Derby winners, it's fair to say the swarm performed at least as well as most experts and polls, and it outperformed many of them since lots of prognostications came up completely empty on this year's Derby.

Unanimous A.I. offered the following chart that shows the confidence the Derby swarm showed in its top choices:

derby-swarm-results.jpg

"As you can see, the swarm guidance's was pretty reasonable, as we had 4 out of the top 5 horses on our board. But, no one saw 32-1 Looking at Lee coming," said Joe Rosenbaum, chief communication officer of Unanimous A.I. "In fact, if you look at the number of correct horses (in the top 4) the Swarm A.I. outperformed the expert handicappers who composed the swarm."

Rosenbaum said its analysis offered the following comparison:

  • Average enthusiasts: 1.6 horses correct
  • Average expert handicappers: 1.6 horses correct
  • Swarm A.I.: 2.0 horses correct

With that in mind, it's still a good idea to consider a group decision-making tool like Unanimous A.I. instead of polls for market research. Just keep in mind that if you're dealing with a phenomenon as unpredictable as horse racing, it can still be chronically disrupted by outliers.

run-for-roses.jpg
Image: Jason Hiner/TechRepublic

Also see

About Jason Hiner

Jason Hiner is Global Editor in Chief of TechRepublic and Global Long Form Editor of ZDNet. He's co-author of the book, Follow the Geeks.

Editor's Picks

Free Newsletters, In your Inbox