DevOps double case study: CloudBees and Perfecto Mobile

Featured Content

This article is courtesy of TechRepublic Premium. For more content like this, as well as a full library of ebooks and whitepapers, sign up for Premium today. Read more about it here.

Join Today

Is DevOps about the delivery pipeline or is collaboration the key? The answer often depends on the companies that make a commitment to it.

What is DevOps? How do different companies approach it? Recently, I conducted a "double case study," asking two separate companies—each fully invested in DevOps—the same set of questions about how they define DevOps, how they work at it, and what it has meant to them and to their customers. An abridged version of that Q&A appears on our sister site, TechRepublic. Now you have a chance to see the two firms' complete take on the subject.

The first company is CloudBees, which provides an enterprise-grade application of the open source Jenkins project, a continuous integration tool written in Java. For this interview, CloudBees was represented by its CEO, Sacha Labourey. The second firm is Perfecto Mobile, a provider of remote access and automated testing solutions for mobile devices. CTO Yoram Mizrachi provided answers on its behalf.

What each company has to say about DevOps is both unique and informative. CloudBees began with a commitment to DevOps, while Perfecto Mobile completed its transition 12 months ago. Labourey prefers a "generic" definition of the term, while Mizrachi's definition is more specific to his company:

Labourey of CloudBees: DevOps is a set of practices and tools that help build an integrated software delivery pipeline, typically automated, from development to production, in order to increase software delivery velocity and reduce friction between teams.

Mizrachi of Perfecto Mobile: To me, DevOps means collaboration that facilitates the ability to release at any given moment. How do we do it? Continuous Quality, a methodology for embedding quality activities into every step of the SDLC [software development life cycle] process.

Summary of interview questions and answers

Initial commitment to DevOps

  • CloudBees: It has been with DevOps from the start in 2010
  • Perfecto Mobile: Its transition to DevOps completed 12 months ago

Enjoying this article?

Download this article and thousands of whitepapers and ebooks from our Premium library. Enjoy expert IT analyst briefings and access to the top IT professionals, all in an ad-free experience.

Join Premium Today

Challenges in transition to DevOps

  • CloudBees: Same as previous answer; just maturation cycles, no major changes
  • Perfecto Mobile: Its three biggest challenges were a change in mindset, developers having to focus more on quality, and operations having to optimize

Definition of "Continuous Delivery," CloudBees: Continuous Delivery is a practice defining how a company, business unit or team will deliver business value through IT in a fast and iterative fashion

Definition of "Continuous Quality," Perfecto Mobile: Continuous Quality is a methodology for embedding quality activities into every step of the software development lifecycle process that is customized to support an organization's specific requirements

Benefit of DevOps to customers

  • CloudBees: It has seen how much more time project teams spend on creating value than on repetitive tasks
  • Perfecto Mobile: Reacting faster to changes and resolutions, overall better product quality

Customer success story

  • CloudBees: Online global travel services firm made switch to Continuous Delivery and deployed Jenkins Enterprise. Cycle times have reduced 75%.
  • Perfecto Mobile: iOS 8 was quite different from prior releases; customers were able to test iOS 8 in beta and work with a real device the day it released.

Customer feedback

  • CloudBees: Customers are seeing amazing results with Continuous Delivery, but more education is needed.
  • Perfecto Mobile: Bug resolution time is down 80%; improvements in each iteration of platform

Main result for company

  • CloudBees: With DevOps and refocus on Jenkins, it has seen rise of developer productivity
  • Perfecto Mobile: Its product is stronger and offers greater value per dollar of client investment

Full text of interview

1. Briefly describe your firm's mission, offerings, and customer base.

sacha
Sacha Labourey (Image: CloudBees)

CloudBees: Our mission is to help companies implement continuous integration and Continuous Delivery with Jenkins. Our first offering, the CloudBees Continuous Delivery platform, is deployed and managed by our customers, typically on-premise, and makes it possible to manage large scale deployments by providing what enterprises typically expect in terms of security, high availability, elasticity, etc.

Our second offering, DEV@cloud, is a fully managed offering, a "Jenkins as a service" if you will, that makes it very easy for teams to perform CI/CD without having to worry about upgrades or infrastructure; it is fully elastic and self-service. Our software offering typically aims at mid, large, and extra-large deployments found in enterprises, while our cloud offering typically aims at single teams within small or large organizations.

Perfecto Mobile: Perfecto Mobile is transforming the way businesses go mobile by making Continuous Quality possible. Perfecto Mobile provides a hybrid cloud-based test lab that supports testing processes earlier and more often in the development cycle, giving way to faster feedback and improved time to market.

Users can access an exhaustive selection of real mobile devices connected to live networks around the world and leverage them for testing and monitoring throughout the mobile application development lifecycle—from development, functional, and performance testing to monitoring and support. More than 1,500 customers, including 50% of the Fortune 500 across the banking, insurance, retail, telecommunications, and media industries, rely on Perfecto Mobile.

2. When and why did your company make a commitment to supporting DevOps?

CloudBees: When we started CloudBees back in 2010, our vision was to make it fast and trivial for companies to create business differentiation through software. We provided a fully integrated end-to-end cloud offering. This fully automated environment was really implementing DevOps best practices and offering those in a simplified fashion to developers and IT. In 2014, we decided to put our entire focus on Jenkins (vs. the full application lifecycle). Interestingly, this increased focus has expanded the type of use cases we now support. Instead of only supporting our own PaaS, we are now supporting many different use cases, in the public cloud or on-premise, from Docker-based deployments to legacy stacks.

yoram
Yoram Mizrachi (Image: Perfecto Mobile)


Perfecto Mobile: About 12 months ago we completed the transition to DevOps. The market compelled us to shift our focus to DevOps as the expectations of the speed of responsiveness increased—new device releases, OS updates, and bug fixes happen on an almost daily basis and we needed to be able to support these changes as quickly as they were being developed and released.

3. Was there a period in your firm before DevOps? If so, what changes and challenges did you face in the transition?

CloudBees: No, we've always been DevOps at heart. Obviously, we went through various maturation cycles, but those were iterations on how we do things, not any major shift.

Perfecto Mobile: Yes, there was a time in which Perfecto Mobile functioned differently, but as the market grew and revved up, we adapted by transitioning to DevOps. Naturally, there were many challenges during the transition. The three biggest were:

  • Change of mindset. It's the movement from developer > quality assurance engineer > deploying through DevOps. This means that the major change is that at any given moment, you need visibility into the product's state and an idea of the level of quality. Teams need to know this on a daily basis because the release schedule is shrinking from every quarter to every month. Developers don't have the luxury of spending two months developing and one month stabilizing the product. Development now takes three to four weeks, and managers need to know exactly where the project stands in order to make real time decisions like go/no-go, feature enablement or disablement, and whether or not a release can be delivered on time. The ultimate goal is to abolish the delayed deadline and instead build a system in which the SDLC [software development lifecycle] is a predictable cycle and accommodates bug testing without taking an excessive amount of man-hours for repair.
  • Developers have to start making quality a much bigger priority instead of submitting hastily written code and relying on QA to catch their mistakes every time. Now that they are part of the quality assurance team, as well as the development team, there's a greater sense of accountability across the board. This translates to all team members becoming better developers who turn out better products. While it seems simplistic, attaching the original developer to a bug does encourage thorough adherence to both testing and best practices, as well as a greater focus on logical architecture and thus, fewer sloppy shortcuts and cut corners that would be costly in the long run.
  • Operations had to optimize. Previously there was a new version of Perfecto Mobile's tool released every four months, they had two months to implement new version with customers, etc. We've since reduced our SDLC to around 24 hours for some upgrades! There was a period in time where installations took longer than the release cycle, which meant we were still upgrading customers to version 5 when we were releasing version 6.

4. What does DevOps mean to your enterprise? In other words, how do you define it and how do you practice it?

CloudBees: Because there is never a one-size-fits-all, I like generic definitions of DevOps. DevOps is a set of practices and tools that help build an integrated software delivery pipeline, typically automated, from development to production, in order to increase software delivery velocity and reduce friction between teams.

Perfecto Mobile: To me, DevOps means collaboration that facilitates the ability to release at any given moment.

How do we do it? Continuous Quality, a methodology for embedding quality activities into every step of the SDLC process—from design through build to production—all based on supporting processes, tools, and testing lab infrastructure that is customized to support an organization's specific requirements. A successful Continuous Quality process optimizes time to market, drives faster and more frequent releases, and enables minimizing escaped defects to production by managing risk in an automated way as early as possible.

How do you do that? Ensure quality throughout the SDLC with an extremely high percentage of testing automation (at least 95% recommended). You must have a stable testing environment that allows for automation. To do this, we moved all of our data to cloud servers, eliminating the server room that we used to have in-house. This was a hard move to undertake but after its completion we had incredible improvements in every department from IT to R&D.

5a. Please define "Continuous Delivery."

CloudBees: Continuous Delivery is a practice defining how a company, business unit, or team will deliver business value through IT in a fast and iterative fashion. CD covers the entire lifecycle, from defining the business requirement to its implementation and delivery to production, its metering, as well as the feedback loop that brings back key metrics to the business, which aims to help refine future iterations. It is important to note that CD covers more than the IT development and deployment steps; it is an overall business practice embedding IT.

Continuous Delivery shares many concepts with DevOps and, as such, they typically happen together. But it is important to know that one doesn't necessarily require the other.

5b. Please define "Continuous Quality."

Perfecto Mobile: Continuous Quality is a methodology for embedding quality activities into every step of the SDLC process—from design through build to production—all based on supporting processes, tools, and a testing lab infrastructure that is customized to support an organization's specific requirements. A successful Continuous Quality process optimizes time to market, drives faster and more frequent releases, and enables minimizing escaped defects to production by managing risk in an automated way as early as possible.

For mobile, having agile-ready processes is a great start, but on top of the agile methods, development teams need fast and actionable feedback from real devices on both the functional aspects of their app and the performance, usability, and security of their app. Embedding all of the quality aspects into each build and delivering constant feedback to the development team gives them early insights into bugs—this is the core value of Continuous Quality.

6. What is the benefit to your customers from your focus on DevOps?

CloudBees: Once projects have to move to DevOps, one of the first realizations is how much more time teams can spend on creating value rather than performing low-value and repetitive tasks, as well as how much faster things occur: from bug detection to product deployment. Past the initial investment to build a DevOps practice, the advantages are very obvious on a daily basis.

Perfecto Mobile: We can now offer faster reactions to changes and resolutions, and overall better quality in our products, which translates to our clients being able to rely more heavily on our system to achieve their mobile app goals. For example, the Galaxy S6 is on our cloud weeks before it's released, which lets clients test their apps on a transparent platform before their end users migrate to it.

7. Please share a customer success story.

CloudBees: A leading online, global travel service completely revamped its software delivery processes. The organization's principal engineer viewed a move to Continuous Delivery as strategic for the company—enabling it to shorten release cycles and get new features into production faster. That move, however, would require a shift in the company's processes and culture.

The team began the cultural change required for CD by documenting existing processes from development through to production and identifying areas of potential improvement such as lengthy test times and frequent patches. The documentation was part of a consistent effort throughout the process to provide visibility and transparency to all stakeholders, enabling the team to understand how they would meet key milestones, which included halving testing times, releasing once per week, halving testing times again, and releasing twice per week.

As part of the move to Continuous Delivery, the company added CloudBees Jenkins Enterprise to the environment to provide a common place for continuous integration and the release of software to production. These steps were vital to shorten delivery cycles and to support the move to CD.

Now, when a developer commits code to the company's Git repository, a Jenkins job runs a build, unit tests, and a variety of automated UI tests, many of which were performed manually previously. Jenkins also creates a snapshot of the latest artifacts, and every 12 hours these artifacts are automatically deployed to the pre-production quality assurance environment.

Availability tests and other Apache JMeter tests are run in the pre-production environment continuously. Twice a week, Jenkins creates a new incremental release, sends a company-wide email notification of the pending release, and then deploys the release to the quality assurance environment, where it undergoes further testing. Finally, the release is deployed to a canary release environment, and then to the entire production environment.

The company has realized several areas of value from the process and cultural changes implemented in order to adopt Continuous Delivery, including a reduction in cycle times of 75%—and its goal is to reduce cycles even further, enabling it to get new functionality into production much more quickly than it did before, with its legacy processes.

Perfecto Mobile: Our customers were able to start testing their apps for iOS 8 in beta and were working with a real device in our facilities the same day it was released. This allowed our clients to update their versions as fast as possible and offer the highest quality to their own customers. iOS8 was very different from previous iOS releases, so several of our customers went through an extended testing phase—which would have been impossible had we not been able to support the new operating system until its final release. We're part of the THEIR Continuous Quality programs, which means they don't have to worry about logistics—we have the device when you need it, every time.

8. Overall, what feedback and results are you seeing with your customers?

CloudBees: DevOps is a pretty obvious step IT teams are moving toward; it obviously takes more time to initiate in heavier organizations, but it is really on everybody's agenda, past, present, or close future. With regard to Continuous Delivery, the companies that have started embracing it are seeing amazing results. However, it requires a lot more education. A number of companies still do not realize how software is eating the world and how their future competitiveness in the market largely depends on how they'll be producing software. Furthermore, Continuous Delivery impacts how business and IT work together, possibly how virtual teams are built and how reporting happens. This requires a true commitment, both top-down and bottom-up.

Perfecto Mobile: Overall satisfaction is increasing all of the time. Objective results—when there is a bug we're able to resolve it immediately which is contributing to customer satisfaction as well. We reduced bug resolution time by 80%, and we're seeing improvements with every new iteration of our platform.

9. Lastly, what has been the main result for your company from your commitment to DevOps?

CloudBees: The inward-facing answer is that before our pivot to focus solely on Jenkins, DevOps, and Continuous Delivery, we had a wide breadth of services, each led by teams with slightly different DNA, each with its own software development practice. Add to this the very distributed nature of our engineering team (we have engineers in more than 10 countries on three continents) and a sense of urgency that makes us feel that sitting down to think about our processes never comes at the right time, and you end up in a place where I think a lot of development teams are. We could clearly feel the output we were getting out of our engineering engine wasn't always representative of the talents we had. As we refocused on Jenkins, we took the time to rethink a lot of things, including how we build and deliver our software, from requirements to production, and build a common practice across the board. In weeks we were already able to see significant changes to our ability to increase productivity and reduce a lot of quite frustrating grunt work. This is obviously an iterative process, but I feel very good about the changes our engineering team has put in place.

Was that a requirement for the business? We nearly tripled our revenues last year and this while refocusing the company solely on Continuous Delivery/DevOps powered by Jenkins. As you can imagine, such a refocus (and a shutdown of our RUN@cloud business) created a lot of disruption, a lot of friction, so we are very proud about this achievement. Continuous delivery and DevOps are clear trends today and many companies have figured out that adoption is the only way to survive in dynamic marketplaces. Adoption is hard, though, for companies that have always followed legacy software development processes. Continuous Delivery and DevOps environments turn legacy processes on their ears. You don't flip a switch and change overnight. It takes tremendous organizational buy-in and a real commitment to change. But the business value is there.

Perfecto Mobile: Our product is stronger and offers a greater value per dollar of client investment, we've expanded our product portfolio, and our R&D department has unprecedented efficiency rates.

Join Premium Today