The tech industry is desperate for AI talent, with some companies forking over $1 million or more to secure the services of those who can code their way to machine intelligence. The problem, however, is that hardly anyone is qualified to evaluate the relative merits of any particular job candidate.

As such, employers are almost certainly overpaying for AI talent, and may not be getting much in the way of talent, to boot.

You (may) get what you pay for

“Google is paying a million dollars for these superstars,” declared Kai-Fu Lee, Google’s former China chief, at the World Economic Forum. “You may not need someone that high, but you’ve got to break the scale for at least one person,” Lee said.

Not surprisingly, it’s scarcity of AI talent that fuels such speculation. As Bloomberg reported, estimates by Element AI put the total population of qualified AI talent at 22,000 globally, with a tiny slice of these (14%) actually looking for work at any given time.

Others, like Tencent, put the number of qualified individuals closer to 200,000 or 300,000 people globally, Bloomberg continued. However, they get to this inflated number through over-inclusion. While Element AI only includes PhD-level folks, Tencent includes a broad array of developers who work on AI-related projects. Given that true AI proficiency includes a unicorn-worthy blend of coding, math, statistics, and data science prowess, it’s likely that exactly no methodology will effectively suss out a particularly accurate way of measuring the AI population.

SEE: IT leader’s guide to the future of artificial intelligence (Tech Pro Research)

Not that this stops enterprises from hiring willy-nilly, anyway.

Sound familiar? It should. It’s exactly what happened in the data scientist market. As with AI professionals, the more generic data scientist requires a blend of domain knowledge, math and statistics expertise, and code hacking skills, as Mitchell Sanders highlighted. No wonder executives told NewVantage Partners that sourcing data scientists borders on the impossible.

For AI, it’s even more difficult, and not simply a matter of finding the financial means to “break the scale.” You can be completely willing to spend gobs of cash, and still not have a clear idea as to what you want to spend it on, exactly.

Who judges the judges?

If AI talent is so scarce, for example, how do you even know what to look for? Good coders, for example, can evaluate the relative merits of an engineering candidate. But if AI professionals are scarce unicorns, there is (by definition) no fellow unicorn competent to evaluate their candidacy. Instead, we’re forced to hire on faith.

SEE: What is AI? Everything you need to know about Artificial Intelligence (ZDNet)

That faith largely comes down to buying into the buzzwords people put on their LinkedIn profiles. To figure out the population of AI professionals, for example, Element AI went to LinkedIn and mined it for people that listed things like natural language processing and computer vision in their bios, or mentioned relevant programming languages (e.g., Python) or projects (e.g., TensorFlow). In other words, the company looked for how people describe themselves, and wasn’t able to figure out whether they’re actually any good with those technologies.

Think about that for a minute. I could put Python proficiency on my resume/profile, but that doesn’t mean I’m actually any good with Python (I’m not). With even junior PhD candidates getting $300,000 job offers, the temptation to inflate one’s experience is great, and the ability to exercise quality control is small.