Businesses typically make it trivial for staff in different departments to share information – requiring them to do little more than click a shared record in a CRM system.
But when different companies exchange information, getting access is often far trickier – with each party often unable to look at each other’s records.
Application Programming Interfaces (APIs) can help companies peer over the wall that surrounds their partner’s systems and access information almost as simply as if it were inhouse.
Taking away the pain of manual processes
UK broadcaster Channel 4 needs to make information available to many different organisations: publications that print TV schedules, ad agencies that want to buy promo slots and companies that distribute Channel 4 content online.
Historically, in many instances, these companies would have had to phone Channel 4, wait for spreadsheets to be emailed or for data to be uploaded to FTP servers.
As well as being time-consuming, these part-manual processes were less convenient for both parties, said James Curran senior project manager in Channel 4’s information systems department. For instance, he explained how rights information had previously been shared with a firm helping distribute Channel 4 programmes online.
“It was a manual operation, in that we would have to write a query to populate an Excel spreadsheet and then email it over to them,” he said.
The broadcaster is in the process of changing the way it shares information with third parties. Using the open-source Mule Enterprise Service Bus (ESB) and integration platform it is building APIs that allow outside companies to programatically query information from its systems.
Channel 4 is building APIs into its scheduling systems, commercial systems timetabling ads, and other software handling issues such as rights management.
Information presented through these APIs is the same as that in Channel 4’s systems and available on demand, said Curran, referencing the API that retrieves programme schedules.
“Any press people can come along and say ‘I need this information and I need it twice a day three times a day’ or ‘I need it up to the minute’. The beauty of it is they can just come and get it whenever they want to, knowing they’re going to get the up to date information,” he said, adding they can choose from a list of formats how they receive the data.
However, building APIs in a way that provides clear and easy access to information held in many different systems sat underneath is less straightforward.
To simplify the process Channel 4 is using MuleSoft tools to build APIs that provide a standardised way to access data from each information domain, for example timetables from the scheduling domain or available ad slots from the commercial domain.
“We’re implementing the Mule interface so the third parties will talk to Mule and Mule will talk to our web services and then bring the information back in a format which they want,” said Curran.
“We’re going to be connecting up our internal systems through Mule, as well as our external systems. So if I call a scheduling service externally, I will go through the same one internally as well. It will reduce the amount of code we have to maintain.”
As well as Mule ESB EE, Channel 4 uses Mule Management Console, a real time monitoring, flow analyser and debugger and Mule Development Studio, a developer IDE. MuleSoft’s tools and support aren’t free to enterprise, Channel 4 pays a licensing cost per CPU core on machines using MuleSoft software.
As well as exposing programme scheduling information via Mule-built APIs, Channel 4 also plans to build interfaces to provide information to ad agencies, as part of its redevelopment of its commercial system.
“That is going to be developed in an SOA [Service Oriented Architecture] type architecture. We’re going to be providing all of these surfaces and to connect all of these surfaces we will go via Mule,” he said.
The multi-year project will allow agencies to view which advertising slots are available, saving them from having to ring the station to check. Eventually agencies may be able to book slots through the interface too.
Along with building APIs to expose data to third parties, Channel 4 is also using MuleSoft tools to streamline information exchange with its disaster recovery site in Buckinghamshire.
In the event of the main site storing Channel 4 content going down, the station would need to run its broadcasting operation from the DR site, so ideally needs to mirror the majority of its content, both programmes and adverts.
The move to using a Mule-built interface to keep track of content back-up is already improving the operation, said Curran.
“We get [notification] on a daily basis content that has not arrived, through Mule querying. We address it and find out why that content has not been delivered to our disaster recovery site,” he said.
“That has exposed issues we weren’t aware of previously and helped us hone in on where these problems lie.”
No change is without difficulties and for Channel 4 one of the first problems with implementing Mule-built APIs across its systems was developer concerns.
“We had to sit with our developers and say ‘This is how we’re going to be doing it’. Some of our developers were resistant to the fact we were bringing in a tool to do our message routing. But I think it’s simply because they didn’t like change, they wanted to continue to do point-to-point integration,” he said.
Channel 4 relies on technical staff based in both the UK and India, and making sure both teams adopted the changes also required effort.
“We had to make sure that they were all working in the same way and same manner. We had to make sure the best practice document was completed. That people understood what we were after. Occasionally someone would go off piste and we had to bring them back. But it’s a real must.”
After various sessions and courses explaining the changes, Curran says the developer team are now on board and realise how the shift benefits them.
“They can pretty much concentrate on the business and .NET development and not worry about this service talking to this service, because it’s the same thing over and over again,” he said.
Other difficulties were technical, such as making sure messages holding database queries and other requests for data weren’t lost in transition.
“We had a challenge with some of the architecture that we were trying to implement, for example, we have an Oracle database that Pirate [Channel 4’s scheduling system] runs on. We were putting message into an Oracle queue, then we wanted Mule to come along and take those messages off the Oracle queue and put it onto ActiveMQ.
“Then obviously we wanted those messages to be persisted into SQL database, so if Oracle went down or Mule went down, or the clusters went down, those messages would always be persisted.
“That was a challenge but that was solved, now we don’t lose any of our messages.”
Messages that have not been handled are placed in what Curran referred to as a ‘dead letter queue’, which will attempt to ensure they are dealt with.