Progressive Release: How to Release Your App With Less Stress

By Blake Sirach

For many companies, software releases are stressful, nail-biting, teeth-gritting stretches of time when the whole enterprise is looking at your team and just waiting for something to go sideways: a major bug on an untested Android device, poor customer response, or unmet stakeholder expectations. But they don’t have to be that way!

Your release process defines how your brand reacts to customer feedback, market changes, and technology shifts. Working with hundreds of clients over the years, we’ve seen just about every kind of release process that exists, and have had smooth (and some bumpy!) releases across every methodology, from pure Waterfall to pure Agile. Synthesizing all of this experience, the WillowTree project management team and I started putting together the defining characteristics of the ideal release methodology, which we are calling Progressive Release.

Progressive Release is steeped in agile, emphasizing nimble iteration, fast response to customer feedback, and lower-stress release periods. It is characterized by a number of tactics designed to de-risk the release process and stay truly customer-centric throughout. I’ve outlined the six pillars of Progressive Release here:

1. Outcomes

Release cannot be an afterthought, something that happens at the end in a mad scramble to the finish. It’s of critical importance for the team to have an established picture of what final success looks like for a product to help inform the design, architecture, implementation, and how the product is rolled out and marketed.

WillowTree leverages an outcome-driven development process to ensure the entire team understands what we are driving toward for every feature and every release. Every feature is tied to a measurable outcome, which contributes to a higher level business goal —so it’s far less about outputs and far more about outcomes. The implementation key for outcomes is adopting a fit-for-purpose analytics suite, e.g. Amplitude.

2. Development Process

To keep the team focused on outcomes rather than wrangling a stage-gated and arduous release process, we leverage a system called Continuous Integration (CI).

CI ensures code is merged regularly to enable automated and lean methods of Continuous Delivery or Continuous Deployment (check this Atlassian blog post for definitions of CI/CD/CD — we often hear these terms conflated, but the differences are important). These key development methods provide for a nimble release cadence of 2-4 weeks without sacrificing robust testing and documentation.

3. Feature Toggles

Further limiting risk while getting the product to market faster, feature toggles allow a team to plan release for specific functionality, to specific user segments, at specific times.

For example, for Regal Cinemas, we leveraged feature toggles to enable fully native ticketing functionality to select markets across the US prior to national rollout. This allowed our team to test this feature in the production environment while also de-risking the rollout of this critical functionality for our client. Feature toggles also empower your team to work on incomplete features which are regularly merged through your CI process, while “hiding” the incomplete functionality from users until ready.

4. Staged Rollout

Staged Rollout enables a single version of a product to release slowly over a period of days, rather than one big-bang (and risky) instantaneous release. This allows your team to closely monitor performance to address production issues before they affect a large number of people. Staged Rollouts also let the team monitor how the product is performing against business goals and success criteria via analytics.

With official support from both Android and iOS (where they call them “phased releases”), Staged Rollout programs are relatively pain-free to stand up. For web, your operations team will likely end up doing a lot of the heavy lifting required for native apps by those services. Staged Rollout for web applications requires a routing or load balancing service that identifies which users to point to the new product.

There are a few different methods your team can leverage to conduct staged rollouts without building new services. For example, the product development team for one of our major media clients incrementally rolls out by platform, then uses another Staged Rollout pattern within each platform from there.

In this example, Android releases prior to iOS, Web, and TVOS using Google’s Staged Rollout service because the Android product has a slightly higher risk profile due to global device fragmentation leading to video playback issues. This method also allows the user acceptance testing team to focus on one platform at a time rather than being completely overwhelmed by performing UAT for uror different platforms at once.

5. A/B Testing

Planning A/B tests in coordination with your release strategy can enable your team to improve the product without going through another point release. Used for data-driven incremental design enhancement, A/B Testing is generally conducted through a third party tool like Optimizely or Adobe Target.

A/B Testing is different than feature toggles in the sense that an A/B strategy should always be in place to incrementally improve the performance of your product, generally around key conversion points, like account creation, purchase, or churn prevention. It’s important to limit the number of variables you test: e.g. if you change an image, the headline, and the background color of the page it’s going to be difficult to determine what variable caused a change).

Where the primary purpose of leveraging feature toggles is to de-risk a software release and test a larger component of an experience, the purpose of A/B Testing is to incrementally improve key details within the functionality and measure their effectiveness against defined product outcomes.

6. Opt-in Betas

Taking it all one step further, we leverage opt-in betas as part of our release program to ensure we are proactively soliciting feedback from specific users. Asking users to opt-in to a beta version of a new release can demonstrate that your company is actively listening and improving their digital services, while also providing incredibly valuable early feedback to the development team.

We leverage a three-pronged approach to data collection during opt-in betas. Using tools like Instabug, we collect crash and bug information for the development team. In parallel, Our Product Research team designs a mixed-method research program including user interviews, diary studies with dScout, and rigorous quantitative validation studies with Qualtrics. Of course, our insights team will be looking at both qualitative and quantitative analytics data to complete the picture of product performance through the beta.