Open Source

4 pillars of Agile integration: Distributed environments, containers, APIs, and people

A panel at the recent Red Hat Summit discussed Red Hat's philosophy and vision for the Agile integration movement, as well as how the organization is helping to drive it.

istock-618460026.jpg
Image: iStock/Rawpixel Ltd

A panel which convened at Red Hat Summit in Boston on May 3rd consisted of the following key personnel:

Ken Johnson, senior director, Product Management.
Steve Willmott, senior director and head of API infrastructure

These executives engaged in a discussion with the audience regarding the Agile integration movement and how it relates to Red Hat endeavors. During the conversation they identified what Agile is all about then enumerated the four pillars behind it and why these are are important.

What is Agile integration?

These days companies need to put things such as business components or services together quickly and respond to market demands. At its base level, Agile integration is a way to get software developed more quickly. The old model involved building software, then figuring out how to tie things together. Now Agile focuses on integration up front, plus the use of smaller teams and an incremental approach — you establish what is required and how to accomplish it more effectively rather than making it into a five year plan.

Red Hat states Agile is a key business indicator for their customers. It's not just about software development but also the acquisition of outside software. The Agile philosophy can also apply to broader concepts, however, such as the infrastructure of integration — how IT overall works to be more responsive — or the changes in the overall tech industry; the driving need to move faster.

SEE: Scrum agile project management: The smart person's guide

Distributed environments

Ken Johnson stated: "integration has been around for decades, but the approach has centered around the technological pattern and the organizational pattern. Both fall short in terms of what modern requirements demand. The technology pattern centered around the ESB (enterprise service bus) which is monolithic; centralized software. All integration logic was centralized and subject to backlog to make changes."

In the past, integration was performed by highly specialized individuals deep within the organization who knew the concept well since that was their main focus. As organizations shift they are transitioning from a centralized approach to a distributed model. These integration experts now participate in Agile teams who conduct this effort as part of their software development. This helps break up the monolith around integration.

Organizations that realize that integration is a key capability for their business will adopt the model to bring integration out of the back office and to the Agile teams. It's a competency organizations need to have to deliver software quickly. Systems and services have to talk together. If you can't integrate quickly you can't deliver solutions quickly.

Steve Willmot stated: "A lot of modern companies have adopted Agile software development and are moving a lot faster than they were. They are adopting organizational models for software like microservices, but they might run into a wall with delivery when integrating with older systems or release models. There's a huge frustration with such barriers. It's important to enable all software teams to do this as part of a team organization."

With distributed integration, you move away from a single ESB and spread it around the company so users can implement their own services. Some technologies make possible what was impossible five years ago. JBoss Fuse and JBoss AMQ are two examples. These are lightweight technologies deployed into an integrated distribution model. These work with the principles of containers and APIs (application programming interfaces) as a deployment packaging and integration model.

Containers

Containers are self-contained application bundles which contain all the necessary components to enable rapid deployment and modification. These are are critical to be able to package internal and vendor software in a manageable and scalable way.

Johnson said: "Containers are a way of packaging your software, and in the context of OpenShift they can be scaled and managed centrally. If you have a consistent integration infrastructure, agile teams can work independently with common libraries and frameworks. This provides an infrastructure that ensures good practices are followed, manageable and governable."

In short, standardization and reusability at the container level are a key element of Agile.

APIs

APIs are the third pillar for Agile. They form a layer across all systems. Many companies are now seeking to implement APIs into core systems that can then be used by everyone. Some organizations have hundreds of APIs. These should be visible to be reused, tracked and secures. On-premise APIs can be tracked, but sometimes cloud APIs aren't tracked as easily.

Willmot said: "We have a lot of customers with a hybrid cloud environment. They have numerous deployments in different elements. Now they have APIs or back end systems in each environment they need to connect together. This needs to be addressable and secure. Customers use 3scale API management to put protective layers in place. Containers manage the back-end systems. APIs are the glue layer - they provide communication, for instance. They can integrate data from different sources to produce one environment. These are no longer edge cases; they are extremely real.

Johnson added: "You may have software or services you own and bringing them all together drives the need to integrate to increase customer satisfaction and a better ecosystem. Every time a customer creates an SaaS implementation that's a new island of information - it's only useful if it can be brought together with other operational data. Customers want to leverage existing investments and enable new initiatives, so expose these to APIs and build new applications to integrate with legacy systems.

People

The key point of Agile is not just the technology but what companies can do with the skills and assets they have. The practice behind the concept requires bringing people together to use the right tools.

As integration becomes pervasive, new people need to participate - not just highly skilled developers, but also key business personnel. The initiative is to bring more people into the integration fold. The cultural mindset is changing across companies, but primarily the focus is on application development. Integration has lagged behind that model of changing the organization to be more agile and meet business needs. Companies need to continue to focus on propagating the right skills out to Agile teams.

SEE: How to apply Agile practices with your non-tech team or business

Willmot: "One of the reasons there is a cultural change happening is that organizations are organizing their development teams differently and they need to get business elements involved. Many used to have a back end coding team and a front end deployment team which were separate entities. There could be a clash between groups. Now it's not just one team delivering a new app."

The panel was asked "What is Red Hat doing to help organizations advance in a technological and cultural mindset shift?"

Willmot: "At the product level we have distributed integration products such a Fuse and AMQ, OpenShift for containers which can provide a management structure and API management. We're working hard together so you'll see integration between our products.

At the organizational level we work with customers to help their teams and drive organizational change via technology. Sometimes we work with partners. Innovation Labs is another concept we're trying to help customers with."

Johnson: "In addition to Innovation Labs and partners, we do one-on-one service engagements; health checks to help organizations see how they have to make the shift, modify teams and use the new technology."

Also see

About Scott Matteson

Scott Matteson is a senior systems administrator and freelance technical writer who also performs consulting work for small organizations. He resides in the Greater Boston area with his wife and three children.

Editor's Picks

Free Newsletters, In your Inbox