Microsoft CEO Satya Nadella has spelled out the technologies the company believes will reshape enterprise.

Chatbots, machine learning, augmented reality and cloud-based automation will be commonplace within businesses in the near future, Nadella told the company’s Worldwide Partner Conference in Toronto today.

During his keynote, Nadella talked about Microsoft’s efforts to help businesses incorporate each of these technologies and satisfy what he said was a desire among CEOs “to use digital technology to change their business outcomes”.

Augmented reality and Hololens

Microsoft HoloLens is an untethered headset that overlays digital information and images on top of the real world around you.

Nadella described the “mixed reality” offered by HoloLens as a sea-change in personal computing, which would transform training within business.

To demonstrate he brought out Microsoft’s general manager for HoloLens Lorraine Bardeen, who showed the audience a demonstration of how Japan Airlines is using the HoloLens to train engineers.

During the demonstration a Japan Airlines engineer showed how HoloLens augmented her vision, placing windows for her email, calendar, web browser, Skype and Power BI dashboard around her, as seen below.

She then demonstrated how HoloLens could show her a 3D model of a jet engine. She interacted with the model, using gestures and voice commands to resize it, to highlight different components and to trigger animated demonstrations and spoken descriptions detailing what each engine part does.

Bardeen said the headset is available now to developers and enterprises.

“I would encourage everyone to look at the applicability of this new medium in the context of everyday business applications, because it will really be the most transformative thing,” said Nadella.

It should be noted that early versions of the headset had a more limited field of view than is suggested by the images from today’s demonstration.

Cloud-based automation

Microsoft wants businesses to use its Azure cloud platform to connect the various collaboration and communications apps, line of business software and professional social networks like LinkedIn, which Microsoft is acquiring.

“What if the software was built in ways that we could connect those disparate worlds,” said Nadella, adding he believes Microsoft is “on the cusp” of being able to do so.

By linking cloud data and services, he said Microsoft wants to allow companies to build highly automated business systems.

The keynote saw a demonstration of how a company called Ecolab is using Microsoft’s technology to automate its business.

The utilities company uses a Power BI dashboard to view real-time data and analysis covering its site management, retail and operational performance. In the example, Ecolab used the Power BI dashboard to see that a customer called City Power had an issue with a cooling tower.

Using a mix of Azure Insights, Cortana Intelligence and Dynamics 365, an alert was generated and a field technician assigned to fix the problem — based on that engineer’s expertise, location and parts available. Each step in the process happened automatically, eliminating the need for manual calls and data entry. That technician could then be tracked using Dynamics 365.

Microsoft also demoed how Dynamics 365 can be configured to automatically perform certain tasks using the automation tool Microsoft Flow, which can link different online services together to trigger actions in response to events. In this instance, Flow was used to trigger a critical email alert and push notification message to certain users, as well as to update a City Power SharePoint list, when a record detailing a critical event was created in Dynamics 365. Ecolab also used the drag and drop app creation tool Microsoft PowerApps to more easily create custom apps for viewing customer information.

“It’s no longer about one monolithic suite and its deployment. It’s this continuous wiring and rewiring of the digital feedback loop, that’s where the power of cloud and the graph of the data in the cloud — underneath all of our applications and tools like Power BI, PowerApps and Flow — come to bear, ” said Nadella.

During the keynote, General Electric also announced that its Predix platform for industrial IoT would now be running on Microsoft Azure.

Machine learning

Artificial intelligence is increasingly appealing to businesses, according to Nadella, with firms finding many uses for the 22 “cognitive service” APIs that Microsoft offers.

These APIs allow developers access to various AI-related, Microsoft services — including speech and language recognition and computer vision.

“Every business process application you can conceive of can be transformed,” he said.

Nadella gave the example of the fast-food retailer McDonalds, which has been working with Microsoft on an automated speech recognition system that transcribes people’s orders at drive-thrus.

In a pre-recorded demonstration the system appeared to be accurately transcribing the speech of a person placing an order. However, the system was not shown in a live demo.

Nadella said that the accuracy of the system stemmed from how tunable Microsoft’s cognitive services are. For instance, in the case of McDonalds, the accuracy of the transcription had been improved by tuning the system to screen out the typical ambient noise at drive-thrus and by teaching it common phrases used when placing food orders.

McDonalds is looking to use the system to push orders directly into its point-of-sale systems, he said, transforming the spoken order into a JSON object the system can parse.

“That type of integration, of cognitive capability into business process, is what we are enabling today,” he said.


Microsoft is also betting that most of us will interact with computers using chatbots in future.

These bots will be able to understand simple typed or spoken questions and commands and respond appropriately, at least that’s the ambition.

Rather than manually launching applications, Nadella talks of information and services being intelligent triggered by these bots, based on our conversations.

Banks and other businesses are already using the Microsoft Bot Framework to build bots for Skype, Facebook, Slack and “anywhere where people are communicating”, he said.

He believes that Microsoft’s virtual assistant Cortana will take on this role of managing services and information for computer users.

Nadella showed an example where Cortana highlighted an upcoming meeting and the people in the meeting, pulling information from their LinkedIn profiles. Follow up questions to Cortana saw the assistant pull in the sales pipeline for the company from the CRM system and make recommendations on certain actions to take during the meeting, for example who to add to which projects based on their skills and experience.

“That’s a new way of how computing is accessed,” said Nadella, adding “over time [it] will fundamentally revolutionize how computing is experienced by everybody.”

Also read…