Stressed computer programmer in front of computer
Image: pressmaster/Adobe Stock

Mux CEO and co-founder Jon Dahl sees a major renaissance underway for applications and web sites based on native video experiences.

“Video is the most powerful medium of communication,” Dahl said. “It’s special because in 10 seconds, you can communicate way more than you can with 10 seconds of reading text or looking at an image. The possibilities of communicating with video real-time inside of your web site or application are endless.”

More than 80% of the Internet’s content is video, and companies are realizing that rich, interactive video experiences in-app are going to drive customer adoption. Real-time video, in particular, is a powerful format with the potential to disrupt an untold number of industries with new front-end experiences.

But online video is a really tough format for developers to work with. That’s a problem, but it’s also an opportunity.

The process for getting video to users

Video at scale may be one of technology’s hardest problems today. As a developer, you might drop an MP4 on S3 and put a CDN in front of it. Easy enough, right?

Well, not really. Next you might discover your users don’t have enough bandwidth, or that your video looks terrible because it’s buffering all the time, or that some percentage of your user devices don’t support the modern codex.

SEE: Windows, Linux, and Mac commands everyone needs to know (free PDF) (TechRepublic)

Indeed, behind every video stream lurks a range of challenging technical problems. Transcoding is complex and compute-heavy. There are many tradeoffs between file size and compute when you’re encoding and compressing it.

If you’re streaming, you have to pick which streaming protocol you’ll use, and which codec you’ll use. Delivering video requires a well-tuned CDN (or potentially multiple CDNs). Even the simple process of  playing back video on devices turns out not so simple: Android, iOS and every web browser are quite different.

Now make that video live, and the job gets much more complicated.

Live video is 10x harder than on-demand video. Don’t believe me? Take a look at this AWS reference architecture and you’ll get a sense for the requisite back-end complexity for delivering and scaling live video.

What’s involved? Well, you have to take the live feed from the camera and turn it into a network protocol. You then send it to the cloud, transcode it, decode it, inspect it and make different compression decisions. You’ll need to store the video somewhere, then you need to use one or more content delivery networks to handle the delivery of the video, with some sort of caching layer to reduce egress. It’s a format with unlimited complexity and new protocols and standards every year.

The video abstraction sweet spot for developers

When developers are bringing video into applications or web sites, sometimes it’s on-demand video (pre-recorded video), sometimes it’s streaming video (one-to-many video) and increasingly it is real-time video (live video). Unless they are video engineers, developers typically don’t understand that these different video formats are inherently different and have completely different requirements on the back-end infrastructure.

Given all the complexity I described above, this is one more decision developers may be ill-prepared to make. But Dahl doesn’t believe developers should have to understand all these nuances in their pursuit of the “sweet spot” for developer abstraction to the video format.

According to Dahl, Mux sees its video API as the right abstraction, neither too high level like the enterprise video hosting providers, nor too low level like the do-it-yourself path that many developers experience through the public cloud providers. Most of their users come to Mux after wrestling with a cloud transcoder, storage platform and CDN, trying to duct tape things together.

Dahl relates Mux’s mission on delivering the right developer abstraction with video to the early days of cloud computing. Rackspace’s approach was very low-level, whereas Google AppEngine’s approach was very high-level. Amazon EC2 then nailed the sweet spot API abstraction to the server that dominated the market and truly made cloud computing take off.

One toolchain for on-demand, streaming and real-time video

Today with the launch of Real-Time Video, Mux may have become the first developer platform to unify all video formats under a single API. That’s big news for developers who need video but don’t want to get a PhD in video delivery.

Unlike streaming video, which is a one-to-many format where three to 30 seconds of latency can be acceptable, real-time video is a one-to-one or many-to-many format, where latency must be 150 milliseconds or less for seamless user experiences. This requires fundamentally different video infrastructure, which is why no one else has brought these technologies together in a single platform.

SEE: Linux turns 30: Celebrating the open source operating system (free PDF) (TechRepublic)

Developer tooling and API abstractions have been game changers for developers over the last 10 years. When you think of a use case as ubiquitous as payment processing, for example, it’s easy to forget that not long ago, it was very unpleasant work to set it all up. Developers had to figure out tens of steps with payment processors, banks and more. But then Stripe came along and divined the right API stack, making payment processing a largely solved developer experience.

For decades, video as a format has been a fundamental part of the Internet, just like text and images. But working with video has been the arena of highly specialized video engineers, so the use cases for native video are only now starting to flourish.

With standard JSON and Rest APIs, webhooks and workflows that allow any developer to work with all video formats, Mux aims to tame video under one coherent API, while abstracting the tough stuff away so that every developer can leverage the power of the video format without having to be a specialist in video engineering.

Disclosure: I work for MongoDB, but the views expressed herein are mine.

Subscribe to the Developer Insider Newsletter

From the hottest programming languages to commentary on the Linux OS, get the developer and open source news and tips you need to know. Delivered Tuesdays and Thursdays

Subscribe to the Developer Insider Newsletter

From the hottest programming languages to commentary on the Linux OS, get the developer and open source news and tips you need to know. Delivered Tuesdays and Thursdays