Innovation

Rise of the robowriters: How AI could crush the spreadsheet

How machines are getting in on the act of writing analyst reports and helping businesses get on top of big data.

While a novel can be a page-turner or a film can keep you on the edge of your seat, a spreadsheet rarely captures attention in the same way.

It takes a human to pluck interesting insights from the columns of figures and to turn raw data into an enlightening appraisal of how to boost a business' bottom line - or at least it used to.

Hoping to apply artificial intelligence to the difficult job of writing analyst reports is Narrative Science, a US firm spun out of research at Northwestern University in Illinois.

Narrative Science made headlines in 2012 for automating journalism. Employed by media outlets such as Forbes, Narrative Science used its software to generate simple newspaper reports on sports and stock market movements - with the firm's chief scientist making the slightly alarming prediction that "90 percent" of news would be written by computers within 15 years.

stuart-c-305x248.jpg
Narrative Science CEO Stuart Frankel.
Image: Narrative Science

In the intervening years Narrative Science has shifted its focus and today earns the bulk of its revenues from writing reports for financial services firms. Companies such as Credit Suisse, Deloitte and MasterCard rely on the automatic reports generated by Quill, Narrative Science's rather anachronistically-named AI software.

"We have developed what we call an advanced natural language generation (NLG) platform. What Quill does is it takes data, primarily structured data, it takes that data, it figures out what's interesting and important and then renders that analysis not in a picture or in a graph but in a natural language document," said Narrative Science CEO Stuart Frankel.

"That document can be anything from a tweet about a sporting event to a twenty-page investment research report that looks like a Wall Street equity analyst looked up a stock, conducted research and wrote up a report.

"Our documents all sound like somebody wrote them."

Gartner fellow Daryl Plummer is bullish on the prospect for "robowriters" like Quill, forecasting that by 2018, one fifth of business content will be authored by machines.

Although there are currently limitations on the sophistication of language that can be generated using software compared to an article crafted by human, Frankel insists the language generation process is far more refined than the 'Mad Libs' approach of slotting data into a template.

"We stay away from the word template because it often implies that a document exists and you're just filling in a bunch of numbers. That's actually not the way that Quill works.

"Quill really figures out what's important and interesting in the data and relevant for the report. Importantly it doesn't write about everything, it will suppress information that isn't relevant."

Based on the required analytics, Quill will test data to see whether it warrants inclusion, for example if cash flow hasn't changed it might choose to omit that whereas if there's been a spike in marketing expenses it may highlight that figure.

"What you wind up getting is a unique document for each company that Quill is writing on."

Taming big data

Frankel sees automated report writing as providing a way of making sense of the vast quantities of data that will soon be available, with 400 zettabytes-worth - one trillion gigabytes - expected to be generated by Internet of Things devices each year by 2018.

"If you think about the growth of data, we're just overwhelmed, and it's growing every day."

Just because companies are handling more data, Frankel says that shouldn't mean everyone has to swot up on statistics.

"I don't know how this got started but there is a belief that everybody in the world need to become data literate, to essentially become a data scientist or a business analyst.

"I think that's crazy, if I'm running a company I don't want everybody to spend their time understanding the nuances of large datasets and becoming an analyst.

"I want them to just have the information they need to do their job and to make better decisions. We feel like what we are doing essentially frees people up to actually do higher value work, rather than spend their time sitting in front of a spreadsheet and writing up a document."

Quill can be applied to any industry but the bulk of Narrative Science's customers today are in the financial services industry, alongside more than 13,000 small and medium-sized businesses and some government clients.

The time it takes to set up Quill to produce reports for a firm ranges from weeks to a couple of months, depending on whether it has been configured to write similar reports in the past. Once set up Quill generates reports from the data instantaneously. In addition, Narrative Science has a number of products targeted at specific needs, such as mutual fund, wealth and fraud reporting, as well as web analytics.

Quill is available as a software-as-a-service offering, which generates most of Narrative Science's revenue, and also as an on-premise version for organisations that don't want to send their data to a third-party datacenter, such as Narrative Science's government clients.

Still a relatively small firm, employing about 80 staff, Narrative Science continues to attract interest from investors, last year raising $10m funding from various outfits, including SAP spin-off Sapphire Ventures. The business has doubled its revenues each year for the past few years and Frankel believes that demand for automated report writing is on the rise.

"What I would reiterate is that this technology is here. It can sound very future-oriented but it's being used by very large enterprises today with lots of success and we just think that it's going to continue to grow."

As machines are used to write increasing numbers of reports, Frankel believes that workplaces will gradually shift away from the spreadsheet and other traditional data-wrangling tools.

"We'll rely less and less on these current tools that have permeated the enterprise, including things like Excel, because the machine will do a lot of that work for you.

"The trend really is moving in our direction. If you look at BI [business intelligence] and visualization companies, many of them are talking about storytelling, this concept of telling a story from your data."

Also see

About Nick Heath

Nick Heath is chief reporter for TechRepublic. He writes about the technology that IT decision makers need to know about, and the latest happenings in the European tech scene.

Editor's Picks

Free Newsletters, In your Inbox