How to stay agile in an integration-driven environment

How to Stay Agile in an Integration-Driven Environment

Software engineering as an industry has shifted away from waterfall software development life-cycle practices to embrace an agile model of iterative development. The added flexibility and collaborative focus of the agile methodology allows for increased efficiency and adaptability during software development discovery so you can streamline and accelerate the delivery of your products.

For teams who have been deeply entrenched in the waterfall mindset, switching to a leaner approach can seem daunting. Phase 2’s approach aims to build trust through transitions with a more tailored way of working with client teams and crafting software. 

 Whether you’re trying to shift from the linear limits of waterfall development or looking for ways to gain momentum with a more iterative process, here are a few ways you can stay agile in an integration-driven environment.

  • Do a feature planning health check. How many of the items in your backlog are blocked by dependent changes? 
  • Stop waiting! A waterfall strategy for integration is blocking work until a dependency is 100% complete or 100% defined.
  • Use the following strategies to stay agile when an integration gets between you and continuously deploying working software.

3 Strategies to Stay Agile in an Integration-Driven Environment

From a technical standpoint, the earlier and more frequently software components interact with one another, the smaller the do-over costs, and more importantly, the less risk for more serious issues later in development. These three technical strategies are great ways for developers to efficiently maintain structure when integrating.

1. Use Release Feature Flags

A release feature flag hides a feature in production from end-users:

if (myFeatureIsEnabled) {
   // show a new button, use a new dependency, perform new logic
} else {
  // behave “the way it did before”
}

For incomplete client-side features, store the value of the developmental feature flag within the code holding the feature.

For external dependencies that are not yet ready, store the value outside the feature code with a server-side flag.

2. Maintain Client-Server Compatibility

To decouple deployments of client and server code, both sides of the integration must be forward-compatible and backward-compatible. 

The boundary between a deployed http client and server forms an implicit contract. When modifying this boundary, think of it like modifying schema in a database migration or function signatures in a SDK library. Is the change an addition? Or would this change break consumers, because it mutates or deletes existing fields?

It is only safe to remove server code after all consuming clients have updated. If client version updates are outside of your control, such as with mobile applications, consider a kill switch.

3. Define Contracts First

Contract-driven development formalizes an explicit contract over an interface boundary:

  1. Whichever team is ready to begin development creates the contract just-in-time.
  2. Propose the contract to the other team in a short meeting and discuss the happy path and edge cases.
  3. Both parties agree to the “set in sand, not stone” contract.
  4. Quickly communicate contract changes by either party as both teams work on the feature and discover new constraints. 

Tips for Contract Writing

  • Avoid using prose / natural language
  • Start low-tech to reduce the barrier to entry
  • Client/Server: write out sample request and response bodies, URL paths, HTTP verbs, and HTTP status codes

Use a Mock Server

To verify that a client appropriately interacts with a contract, consider adding a mock server.

A mock server should be so trivial to maintain that it unblocks maintainers of client code from feature development, rather than adding calendar time to an integration.

How many times has the server not been ready for demo day? Consider retroactively adding existing endpoints and making the mock server a first-class environment like staging. Then you can perform client-side software development or stakeholder demonstration uninhibited by surprise server outages. 

Use Contracts Internally

Use contract-driven development at different scales to achieve decoupling and parallelization of sprint tasks.

You can easily identify a client-server boundary and additionally draw boundaries from a single application.

Case Study: Parallel Development with Feature Flags and Contracts

Consider a client application (such as a mobile app or javascript app) to divide into a data layer, business logic domain layer, and UI layer. 

The work for a new feature could be decomposed into multiple UI tasks and data transfer tasks. 

How can you complete and merge the UI tasks before the data task(s) are complete, or vice versa? Follow these 5 steps.

1. Feature Request

As a user, instead of seeing my last order (existing behavior), I want to see all of my previous orders. I also want to see my favorite order, if I have one.

Existing UI:

// Home Page
onPageLoad() {
  domain.getPreviousOrder()
    .then(order => {
        createPreviousOrderWidget().display(order)
    })
}

Existing Domain:

DomainInterface {
  getPreviousOrder(): Optional<Order>
}

DomainClass: DomainInterface {
  getPreviousOrder(): Optional<Order> {
    return data.getPreviousOrder()
  }
}

Existing Data:

// Data
getPreviousOrder(): Optional<Order> {
  httpService.getSingleOrderIfExists()
}

2. Create Domain Contract

The teammates working on the data layer and UI layer form a contract on the domain layer. They agree the final version of DomainInterface will be:

DomainInterface {
  getPreviousOrders(): List<Order>
  getFavoriteOrder(): Optional<Order>
}

3. Complete UI Task In Parallel

The teammate(s) in charge of the UI layer delete the DomainInterface.getPreviousOrder(): Order function as it becomes unused on their branch. Then they write a minimal stub in the domain layer, hidden behind a feature flag, to unblock their work from data changes:

DomainInterface {
  getPreviousOrders(): List<Order>
  getFavoriteOrder(): Optional<Order>
}
DomainClass: DomainInterface {
  getPreviousOrders(): List<Order> {
    return new List() { data.getPreviousOrder() }
  }
  getFavoriteOrder(): Optional<Order> {
    if (featureIsEnabled) {
      return new Order() {
        items = “2 apples, an orange, and a pear”
        total = 6.04
    }
    return null
  }
}

The previous orders are now showing as a list, and the favorite order with stubbed data only displays with the feature enabled:

// Home Page
onPageLoad() {
  domain.getPreviousOrders()
    .then(orders => {
        orders.forEach(order => {
          createPreviousOrderWidget().display(order)
        }
    })

  domain.getFavoriteOrder()
    .then(order => {
        createFavoriteOrderWidget().display(order)
    })
}

A code reviewer can turn on the feature flag to see the new UI.

The UI changes are complete and safe to merge into the codebase before or after the data changes.

4. Complete Data Task In Parallel

The teammate(s) in charge of the data layer keeps DomainInterface.getPreviousOrder(): Optional<Order> and updates its implementation to avoid breaking existing UI. Then they connect the new functions to new data layer functionality:

DomainInterface {
  getPreviousOrder(): Optional<Order>
  getPreviousOrders(): List<Order>
  getFavoriteOrder(): Optional<Order>
}
DomainClass: DomainInterface {
  getPreviousOrder(): Optional<Order> {
    return getPreviousOrders().maxBy(order => order.dateOrdered)
  }
  getPreviousOrders(): List<Order> {
    return data.getPreviousOrders()
  }
  getFavoriteOrder(): Optional<Order> {
    return data.getFavoriteOrder()
}

They update the data layer with their changes:

// Data
getPreviousOrders(): List<Order> {
  httpService.getAllOrders()
}
getFavoriteOrder(): Optional<Order> {
  httpService.getFavoriteOrderIfExists()
}

At this point in version control, the data and domain application code for getFavoriteOrder() is unreachable by application code. If the UI changes are not complete by the time this enters code review, it is not obvious how to review it. 

One solution is to include a code snippet as a PR comment showing example execution of the code, perhaps printing the domain results to a log. Apply the change locally for the sake of manual testing. 

Regardless of the difficulty in manually testing temporarily unreachable application code, the domain and data layer should have thorough automated testing.

The data changes are complete and safe to merge into the codebase before or after the UI changes.

5. Address Merge Conflicts

Whether merging the data layer or the UI layer first, the conflicts will be at the domain layer.

Below are the conflicts that would occur if the UI layer was to merge first. 

In most cases, discard the UI teammates’ modifications since they are a stub for the data logic. Retain the deletion of getPreviousOrder(): Optional<Order> because it is no longer in use.

All data teammates’ changes can be kept except for the changes to  getPreviousOrder(): Optional<Order> since it was deleted.

DomainInterface {
  getPreviousOrders(): List<Order>
  getFavoriteOrder(): Optional<Order>
}

DomainClass: DomainInterface {
>>>>>>>>>>>> UI
getPreviousOrders(): List<Order> {
    return new List() { data.getPreviousOrder() }
  }
====
  getPreviousOrder(): Optional<Order> {
    return getPreviousOrders().maxBy(order => order.dateOrdered)
  }
  getPreviousOrders(): List<Order> {
    return data.getPreviousOrders()
  }
<<<<<<<<<<<< Data

>>>>>>>>>>>> UI
  getFavoriteOrder(): Optional<Order> {
    if (featureIsEnabled) {
      return new Order() {
        items = “2 apples, an orange, and a pear”
        total = 6.04
    }
====
  getFavoriteOrder(): Optional<Order> {
    return data.getFavoriteOrder()
<<<<<<<<<<<< Data
}

The conflict resolution yields executable code because DomainInterface is in the final state agreed upon by the contract, and the conflict-resolved  DomainClass matches the interface’s function signature.

Conclusion: Simplifying Integrations to Reach Agile Business Goals

To solve complex business problems, software components must integrate with one another. A waterfall approach can put dependent code in a standstill at best and all code at a deadlock at worst. Strategies like feature flags and contract-driven development allow software teams to collaborate on an integration on an agile timeframe and empower technical teams to move forward on business goals.

Integration boundaries will not be entirely defined until all of the code is written. At Phase 2, we carefully consider how to smooth out bumps in an integration road. We are intentional in our approach to the development process to start earlier on the quick pieces without waiting for the slow, hard pieces to be entirely complete. With these tools, we remain agile in an integration-driven environment.

Here are some other Phase 2 blogs you might find useful!  Using Automated Testing and How to Write a Valuable Test Suite


Let’s Transform Your Next Technology Phase Together | Phase 2

For over two decades, Phase 2 has built custom world-class software solutions for large-scale enterprises, well-funded startups, and more. We have refined our process to leverage our expertise, and our deep bench of elite software developers and creatives work to find the best solutions for clients. 

Phase 2 is a trusted technology partner with a deeply ingrained culture of collaboration and cooperation. Whether you’re looking for a bespoke software solution for your next technology phase or the opportunity to join a driven and dedicated team of top-notch talent, contact us or check out our Careers page.