Blog header with title "Using Automated Testing to Confidently Release Software" by Emily Pleimeier, Software Engineer. The background is a dark navy, the text is white and there are lime green, stylized lines of code coming into the graphic from the right. On top of the code, there is a white circle with a royal blue cog in it, and a checkmark in the center of the cog.

April 6, 2021

Using Automated Testing to Confidently Release Software | Phase 2

Do a technology health check… when was the last time you released an update of your software?

As you well know, releasing software is not an easy task.

Picture this:
Your team has spent months working on new features and fixing bugs. Now, it’s release day.
You hit the big red button to release your update, hold your breath… then comes the bad news.
Your software released with a bug that must urgently be fixed with a follow-up release.

Does this sound familiar?

At Phase 2, we don’t fight the fires caused by releasing our software. We prevent them. Our secret sauce: automated testing.

An automated test is a piece of software that runs against your software and without human intervention can determine whether your software behaves as expected.

So why testing? Why should it be automated? Is automated testing within budget? Let’s walk through why automated testing is a valuable tool for your organization.

Automated tests catch bugs

Bugs are inevitable for even the most seasoned developers and can slip past the most stringent QA testers. Bugs are technical debt imposed on your next release. If your team introduces a bug in one release then you must reserve capacity in a future release to fix the bug. Automated testing reduces the number of bugs your team introduces with a release.

The less bugs you commit to per release, the more features you can add and provide your end user with a more robust experience.

Automated tests document software

A behavior-driven test follows the structure: “Given some precondition, when some action occurs, then the expected result is…”

Infographic depicting if/then formula for behavior driven tests. A blue diamond with a 1 in it has "if condition" labeled underneath. An arrow to the right of it points to the right to a pink box with a 2 in it. This has "when action" labeled underneath. Another arrowing point to the right goes to a green circle with a 3 in it and "then result" labeled underneath.

A series of tests written around a function come together to describe its intended purpose and behavior. Tests serve as living documentation in that if the software changes, the tests must adapt to verify the changed behavior.

Not every team member will know everything about your software. Over time, knowledge silos will form if developers continuously work on the same section of code. Tests reduce knowledge silos by allowing the original developer to describe to future maintainers what specific functions accomplish.

Code ownership is disseminated across the team.
Tests give rich, up-to-date documentation at the fingertips of anyone who maintains the code. This puts team members on a more even playing field when it comes to knowledge about sections of the codebase.
When knowledge silos are broken down team members are empowered to view the code base as everyone’s code. Suddenly it doesn’t make sense to have one person working on the same section of code.
Adding a team member is less risky.
Our most recent team member was entirely new to the tech stack and the client’s problem domain. Thanks to an expansive test suite, he was able to hit the ground running.
Without the intervention of another team member, he could run the tests against his changes and immediately know if what he did broke existing functionality.

Automated tests protect business value

What is the financial impact of your software’s bugs? For public-facing apps, broken software can lead to a decrease in conversions and retention. For internal apps, broken software can decrease employee productivity.

The duration of how long a bug exists in production can dramatically increase the cost of doing business. Some bugs live in a code base for a long time undetected.

For example, if a bank’s software goes offline due to a bug, the outage could cause the bank to lose money the longer the system is down due to whatever action the bug is hindering (fewer transactions happening, fraud not being caught, etc.) If the bank doesn’t even notice the outage, then it’s likely going to be down for a long time and the bank loses business opportunity.

To fix a bug, it must first be identified, then introduced into the sprint, repaired by an engineer, then QA must be completed before the software is released.

Even after the software is released, there is a period of time before the user adopts the new version containing the fix. One risk of not being timely with bugs or not utilizing automatic testing to help circumvent bugs is that some users simply do not wait for the fix and uninstall the software or abandon use.

Automated tests continue to provide and build value

Quality assurance can become a time sink. A manual test performed by a team member is valuable only for the version of software that is currently being run.

Once an automated test is written, it continues to provide value because it can run on any subsequent version of your software. Most automated tests run in fractions of a second.

Automated tests protect new features and verify that old bugs are not reintroduced. While tests take time to write, they are an investment in the longevity of your software.

Is Automated Testing for You?

Automated testing allows software teams to shift focus from fire fighting to fire prevention. This gives your team the capacity to dream big in implementing new features. You will ease the pains associated with scaling your team. Tests will catch broken software before it is released into production, saving you and your team time, money and probably some sanity.

Tests allow you to release your software confidently. If the response to released software is quiet, it speaks volumes. With the mitigation of risk of time and cost of talent, who doesn’t have budget for automated testing?

At Phase 2, we pride ourselves on releasing software regularly, confidently, and quietly. We embrace a culture of incremental improvement. The shift to automated testing does not happen overnight.

What is the next step you can take towards a more confident release?