iPhone

Why Apple is about to build, buy, or partner on a web search engine

Apple's Siri could change the game in search and put a lot of pressure on Google, but first Apple needs to deal with web search integration.

Apple's iPhone 4S was a disappointment to all of those who were expecting a redesigned iPhone 5, but in the grand scheme of the things the launch of the iPhone 4S may turn out to be Apple's Chamber of Secrets.

Forgive the Harry Potter reference, but Chamber of Secrets is the second book in the seven-book Harry Potter series, and while it's generally the least favorite of the books among Potter fans, by the time you get to the final book you realize that Chamber contained critical plot information that foretold important future events.

The fact that the iPhone 4S was an incremental hardware upgrade and lacked a new design has largely overshadowed its one revolutionary feature that could shape Apple's future: Siri voice commands and voice-activated search.

Apple has limited Siri to the iPhone 4S to start, but that probably has less to do with Siri needing extra computing power on the phone and more to do with Siri still being in beta. Since Siri requires a cloud connection, limiting Siri's spread at first gives Apple the opportunity to stress-test its data centers and scale up for the future.

Even with its beta quirkiness, Siri is impressive. While Google Android and Windows Phone 7 both had a jump on the iPhone in terms of voice control, Apple has zoomed past both of them with the purchase of Siri and its integration into the iPhone. The big deal for Siri is that it understands natural language and it is standardized across a lot of different applications on the iPhone. The user doesn't even have to be aware of which apps to use. You can simply give Siri a natural language command and she automatically interacts with the right app to execute it. That's a nice step forward for voice user interface (VUI).

The Siri experience hearkens back to the launch of the original Macintosh in 1984 when Steve Jobs climaxed his unveiling by saying, "I'd like to let Macintosh speak for itself" and it did (using Macintalk software), which blew the minds of techies at the time. Of course, in a larger sense, the whole thing also points back to the computer in Star Trek and its VUI. In other words, Apple has been entranced by the idea of integrating speech into everyday computing for a long time -- almost from the beginning of the company.

However, as fun as it is to bark orders at your phone and have it obey your commands in real time, the revolutionary piece of Siri is what it does in Internet search. It's early and Siri is still imperfect, but there are moments when Siri drastically streamlines the search process and gives us a peek at the future.

For example, I recently asked Siri for "the closest Mediterranean restaurant" (right) and got a list showing 11 restaurants, their user ratings, and their distance from my current location. Clicking any of the selections in the list immediately took me to a map.

Another time, I asked Siri, "How many calories are in a kiwi?" She came back with 46 calories along with a full chart of all the nutritional information for a kiwi.

Last week when I was doing research for my article iPhone and Surface: The moment Apple and Microsoft diverged, I got frustrated trying to find historical data on the market cap and revenue of Microsoft and Apple going back to 2007. In desperation (and half-jokingly) I asked Siri a question about Microsoft revenue in 2007 and surprisingly got an answer, based on data from Wolfram Alpha (which was also the source of the kiwi data). That eventually led me to Wolfram Alpha on the web (from my computer) to do a full lookup of the data, but the fact that Siri led me there was a big "ah ha" moment.

Siri can also help you find nearby physicians, lookup movie times, and pull up weather data when you ask questions like, "is it going to rain tomorrow?" Siri still has a hard time understanding normal speech at times and it's limited by its access to freely available data sources like Google, Wolfram Alpha, and Yelp. But, Apple has shown us what's possible with a much more approachable VUI than anything we've seen so far in the consumer market. Siri is almost like an IBM Watson for the masses.

One of the important things to notice about Siri is how it disintermediates search results pages in general and Google specifically. Instead of giving you a page of possibilities to choose from, Siri tries to give you a single authoritative answer to your question. Since Google makes all of its money by allowing advertisers to place their ads next to the items listed on the search results pages, it's easy to see why Google Chairman Eric Schmidt is talking about Siri as a competitive threat.

The next step

Now that Apple has opened the door to a natural language VUI and demonstrated new possibilities, the game really begins. Google and Microsoft will undoubtedly take cues from Siri and bring similar functionality to Android and Windows Phone, since both companies already have a lot of engineers working on voice technology. That means Apple is going to have to rapidly improve and innovate Siri if it wants to be a leader in VUI. Siri has two areas that need the most work: 1.) it needs to keep improving voice recognition, and 2.) it needs more data sources to feed Siri and integrate into its equation.

Currently, if Siri doesn't have an answer to something, the fallback is to throw the question to a standard mobile web search. That's not going to suffice for long -- especially when you consider the level of integration that Google and Microsoft will be able to do since they both own search engines. Siri needs a web search that is tightly integrated into the service in the same way that Wolfram Alpha and Yelp are today.

That leaves Apple with three options: build, buy, or partner.

Build

Siri itself is already a bit of a search engine, and with all of the searches that are now happening through Siri and running through Apple's servers, the company is amassing a treasure trove of data about the ways people are using voice search. Plus, all of the Siri data is tied to specific users and that will give Apple an excellent opportunity to do personalized search in the future.

Last year at the D8 conference when Steve Jobs was asked about Apple buying Siri and going into the search business he said, "They’re not a search company. They’re an AI company. We have no plans to go into the search business. We don’t care about it — other people do it well."

While Jobs has famously denied lots of things that Apple eventually went on to do, it's hard to see Apple building its own web search engine from scratch based around the core team it acquired from Siri. That would take years and lot of resources. Just look at how much money Microsoft has had to throw at building Bing, with only moderate success and no hope of turning a profit any time soon.

Buy

The faster on-ramp for Apple would be to buy one of the smaller players in web search, integrate it with the Siri team, and put most of its resources into customizing a VUI that feeds Siri. There are a few decent candidates that Apple could gobble up: Blekko, DuckDuckGo, Yippy, Dogpile, and even good old AskJeeves.

Apple has $80 billion in cash reserves so it has plenty of resources to buy any of these search engines. The best options would likely be DuckDuckGo and Blekko. Both of them already do some things better than Google, but don't get much attention because they're so small.

Partner

If Apple were to partner with another company in search it would have to be Google, Microsoft Bing, or Yahoo (which has mostly abandoned its own search for Bing). Google is an obvious "no" since it's Apple's archrival in mobile. Bing might look like it makes sense in the short term, since Microsoft has fashioned Bing as a "decision engine" rather than a search engine and that fits pretty well with what Siri is trying to accomplish.

But, Microsoft is destined to want to do something similar to Siri in Windows Phone and that will be enough to scare Apple away from a doing a deal with Microsoft.

Sanity check

With Siri, Apple has lowered the friction on search and turned it into a mellifluous experience. But, to take it to the next level, Apple is going to need much tighter integration with web search. Building a search engine would take too much time and there aren't many good options for Apple to partner with in search, so the most likely scenario is that Apple will buy a smaller player and integrate it into Siri.

Siri clearly has tremendous future potential for Apple across its entire product line. By the end of 2013, I expect that we'll see Siri on most iOS devices and Macintosh machines. Nick Bilton even believes Siri is the revolutionary interface that Steve Jobs wanted to bring to television sets.

The bigger and more entertaining question is if Apple does jump into search with both feet, will the company freely release Siri on the Web and challenge Google directly? I doubt it, given Apple's affinity for hardware/software integration, but it's fun to consider, especially as we look at Apple's new VUI as arguably the most important new development in search in the past decade.

Also read

About

Jason Hiner is Editor in Chief of TechRepublic and Long Form Editor of ZDNet. He writes about the people, products, and ideas changing how we live and work in the 21st century. He's co-author of the upcoming book, Follow the Geeks (bit.ly/ftgeeks).

32 comments
abiyotb
abiyotb

VUI will never replace touch related interfaces because there is a special pleasure in touching (except is a specialized specific situations). It is just like choosing either to "talk" with or to touch the thing that u like.

Gisabun
Gisabun

Maybe I should trademark these names so Apple doesn't grab them [unless they are already taken], iSearch, iFind, IBiased, iCostMoney, iProprietary Apple will then force all Apple products to use their search engine even if you choose another one [say in your browser]. [Then, of course, Apple rushing to get it live, will allow so many security loopholes that the scammers and virus makers will think it's Christmas for a month]. :-)

aflynnhpg
aflynnhpg

If Apple actually came out with a search engine it could be a risky move especially if they locked users into using their search engine. The vast majority of internet users like googles search engine, and phones that have locked people out of google have lost sales as well as encouraged people to hack their phones. Apple doesn't like people jailbreaking their devices. This type of move could actually backfire on them.

stevencel
stevencel

Jason, Isn't Safari (Apple product) a search engine?

TNT
TNT

I agree with you Jason, and your article is spot on. I look forward to Microsoft innovating a whole new paradigm of interaction with Kinect for Windows. It will not only have Siri-like voice commands but possibly make the mouse irrelevant with gesture control for desktops. Search engines need to think about the future and prepare for voice input, regardless of what apple does with its own service. The future is almost here and I look forward to seeing how Microsoft's implementation stacks up against Apple's.

kiwidave
kiwidave

I'll bet a kiwi has more calories than that! http://en.wikipedia.org/wiki/Kiwi I think you mean a kiwiFRUIT! :) While voice recognition technology will be (has been?) quickly adopted in the mobile space, I can't see this being as heavily used at the desktop, especially in open office environments. I can see some possible humorous unintended search results, though. It takes me back to when I first got my hands on a Newton. Until it was trained, it came up with some bizarre interpretations. A colleague of mine tried writing his name (Peter Moodie) and it responded with "Peter Must die". I fully agree with CaptainWhizz. Ignoring the voice recognition aspect for now, the best thing about the underlying technology is its use of divergent and specialised sources of information, that can provide much richer and more relevant results. This is the future of search technology and Google needs to understand this if it wishes to maintain it's position.

techconc
techconc

"Apple???s iPhone 4S was a disappointment to all of those who were expecting a redesigned iPhone 5" Correction: The iPhone 4S was a disappointment to the few dopes who happen to both believe every rumor they read and who chose to ignore Apple's recent history of reusing the same form factor for more than one generation of devices. The rest of us were actually quite impressed with the 4s from the beginning.

CaptainWhizz
CaptainWhizz

The reason we use Google and the like is that they are web crawlers finding anything remotely relevant, however the data is dirty and unverified and is thus of little use. Also of no use is the boast that it found 8,247, 856 (ir)relevant entries in 0.2 of a second. Siri goes straight to the heart of the problem, it KNOWS it is a request for a restaurant and automatically goes to dedicated sites that can answer the question SUCCESSFULLY, without giving you the other 8,247,843 irrelevant results. This in time, coupled with dabbling in non-core specialities that they know nothing about (phone software and hardware) will ultimately prove the undoing of Google. A search request, such as Google knows not who you are, where you are, what age and sex you are and more importantly why you are (re)asking this question. Siri is just going to build up a lot of relevant content providers that CAN ANSWER the question and go straight there. We don't go to a butchers and ask for a recommendation on central heating, why should we ask an unqualified person for an expert opinion on something he knows nothing of? Web search has not changed a jot since the dawn of the web (excepting the delivery of quicker irrelevant results) - just the same crummy list. The opportunities lie with the content providers to do more than just feed Googles invoicing machine. Top Table and Wolfram Alpha are but the start of this trend. This is going to be the content equivalent of iTunes and the music/movies providers and everybody by the search engines will become a winner.

Olderdan
Olderdan

I seem to recall a quote from a whimsical Steve Jobs telling people they'd be amazed at how they interact with the then unannounced and unseen iPad. About 4 trillian keyboard designs (including keyboards on the back) made the headlines. But I think Siri was what he had in mind all along.

emstutzman
emstutzman

Microsoft has had voice commands on the phone and PC for years now. So, are you saying you are excited that Apple is finally catching up in this area?

jmattor
jmattor

I am truly sorry to say that Siri has been a big disappointment to me. On numerous occasions I've had to stop and pull over while driving to turn Siri off so that I could switch to "Voice Control" to make my phone calls. Voice Control works without hesitation every single time. Siri doesn't. 8 out of ten times I'll get a response indicating it cannot perform that function. Or when I ask it to call a number of a business in my address book it instead searches the Internet and gives me the address. Apple needs to do a better job insuring connectivity to it's servers so that Siri might at least connect half the time.

AGOlbert
AGOlbert

Another great article Jason! The fact that Siri disintermediates every app on the iPhone is a powerful, maybe risky, business action by Apple - not just cool technology.

doctordirt11
doctordirt11

I guess all the advertisers will have to pay a 30% commission on sales.

johnbasslosangeles
johnbasslosangeles

I didn't realize the long term implications of Siri, but of course, I think your assertion is correct. I just bought a 4S, and by far the most amazing thing is Siri. Its ability to actually interpret English is remarkable, and scary (in a good way.) In addition, I'm tickled by the way that it makes "assumptions" based on my previous behavior (like Google search results.) When I ask Siri to call "Joe," even though there are four "Joes" in my phone, it calls the one I talked to most recently. It is certainly my first experience with a technology that smacks convincingly of artificial technology. Correct me if I'm wrong, but Apple didn't develop Siri, right? Didn't they purchase it? http://www.johnclassick.com

nv1z
nv1z

I do not know what it means, but SIRI backwards spells IRIS. I wonder if eye-scan technology will find its way into phones for one purpose or another. Shades of "Minority Report."

jred
jred

Safari is a web browser.

Frenz9
Frenz9

I think you took that sentence out of context, you cannot argue that is was a disappointment for a lot of users when viewed from what they were expecting but they were happy with the phone. I believe the 4s main purpose is for the future, a phone for BETA Siri, imagine what they could do with the iphone5 when siri has advanced a year or so.

dogknees
dogknees

They had invented a brain scanner that operates via a browser. Otherwise how would they determine "why you are (re)asking this question"? Half the time I don't know the real reason I'm asking a question. There's a lot of sub-concious stuff going on as well as conscious thought.

deasystems
deasystems

Siri is not "voice commands." Siri is a voice interface based on AI-backed speech recognition. Apple is not "catching up;" it has leapfrogged past voice functions on all other platforms. (By the way, you should be aware that Apple had "voice commands" built-in to ALL its computers since the introduction of PlainTalk in 1993.)

victorpanlilio
victorpanlilio

http://en.wikipedia.org/wiki/Speech_synthesis#Apple "The first speech system integrated into an operating system that shipped in quantity was Apple Computer's MacInTalk in 1984. The software was licensed from 3rd party developers Joseph Katz and Mark Barton (later, SoftVoice, Inc.) and was featured during the 1984 introduction of the Macintosh computer. Since the 1980s Macintosh Computers offered text to speech capabilities through The MacinTalk software. In the early 1990s Apple expanded its capabilities offering system wide text-to-speech support. With the introduction of faster PowerPC-based computers they included higher quality voice sampling. Apple also introduced speech recognition into its systems which provided a fluid command set."

smartin007
smartin007

Where is this on the PC? I need to turn this option on and give it a try.

joecamaro
joecamaro

Like don't drive and yack on the phone at the same time

belli_bettens
belli_bettens

Now we just have to wait for some malware to hook into the system, triggered by specific search queries, and start up some malicious code...

deasystems
deasystems

Apple has spent the last 18 months extending and further developing Siri since it purchased the company that originally created it. So, the answer is that Apple purchased Siri AND continued to develop it.

Gisabun
Gisabun

Siri was bought a few years back. I guess Apple toiok the lazy approach and just bought a product. I guess they don't really invent anything in Cupertino.

vendors
vendors

If I can't understand him the how many times did he have to retrain the phone (SIRI) to get it right. GET REAL! It just doesn't work the first time. You need to train the service.

vendors
vendors

In 1982 the TI 99 Business computer has speech. How does Apple get credit for SIRI since it was it's own entity? Siri is the pioneer right?

roger that
roger that

In 1992, I began to use voice control to open application on my Mac. In 1994, I wrote a program that used voice recognition to take orders over the telephone for my plants. It worked pretty well, but when it didn't understand a series of words, it always opened Quark Express. So I had to abandon it. Still, many orders were taken and filed in my database correctly.

GeoffMichael
GeoffMichael

Judging from their sales, it appears to be just the "stuff" that consumers seem to want and purchase, eh?

sperry532
sperry532

As does Mircrosoft, as does Nuance, as does almost every hardware, software, manufacturing, retail, service, and every other type of company in the world. If someone has a product, process, or service that a company finds of value, they acquire it (and the ancillary patents and copyrights) if they can. It's usually more cost-effective than starting from scratch. And it has a lot to do with not re-inventing the wheel.

Editor's Picks