Sometimes, developers go to war over what type of test to write or what techniques to use. But they often miss the forest for the trees by not asking, “Why do we write tests?” The answer to that question guides our actions and thinking. So we want to answer the question, why do we write tests?
The reason is quite simple:
to increase confidence and,
to provide documentation.
Losing sight of those reasons gets us lost in the details. Tests should provide clear documentation and to provide confidence that the system works as intended. The reality is that in the pressure and desire to grow the features of our software, we can lose sight of this fundamental purpose, even if we've bought into these primary testing objectives. The test we wrote on Day 3 that gave us confidence and documented our intent may be slowing us down or confusing us on Day 33 or 133.
Automated testing is code written alongside business logic that executes scenarios and validates that functionality is correct. It’s not a replacement for manual testing but a way to constantly ensure the evolving software is healthy and high-quality.
The RoleModel Way of Testing is how we approach software testing in our projects. We’ve developed these principles and techniques over hundreds of successful projects that have resulted in sustainable software assets. Writing tests is fundamental to our focus of Iterative Value. Without it, we don’t have a solid foundation to build on, and we can’t work iteratively with confidence.
Why is confidence so important when you’re developing software? When you don’t have confidence in your software, you become afraid of it. We’ve seen this many times as consultants when we come into software projects without tests, and the software has deteriorated over the years. It gets to where it is very hard to build new features. Every time you do something, it breaks, and the cycle repeats. You fight against that deterioration in software by continually improving the software, refactoring it, and making it better. Tests enable you to make a change or improvement, and then you can run your test suite and know if you broke anything.
Having confidence in your test suite means ensuring that your tests and the overall test suite are:
correct, and actually do what they claim
consistent, no intermittent or occasional failures that erode your confidence, and
provide a tight feedback loop significantly faster than manually testing each change you make to the system.
Though Correct and Consistent should be obvious, they often get so much focus that the Tight Feedback Loop part gets lost. A bug is found, so a test is added. An intermittent test failure is found, so we add some protection around it (e.g. sleep(1)). Before we know it, the test suite goes from seconds to minutes and perhaps from minutes to hours.
At RoleModel, a large part of confidence comes from having feedback along the way. This applies to many areas, but especially to testing. The faster our feedback, the bolder our changes can be. A fast feedback loop lets you know if you broke something fundamental. And you can receive this valuable feedback without having to break your stream of consciousness.
Receiving quick feedback from tests means a developer should be able to:
make a small change (such as modifying a class or method) and get feedback in seconds
refactor a sub-system and get feedback in less than a minute
run the whole suite in a few minutes
The important factor is more than how fast the whole test suite takes to run. For a large system, it might be understandable to wait 15 minutes for a full run. However, if you need that full run for 80% confidence in your change, then something needs to be fixed with the testing approach. A test suite that takes too long to run will decrease confidence because the whole suite rarely runs. You might work for an hour or two without running the full range of tests because you didn’t want to break your rhythm of development. Then, when you finally do run the suite, it’s harder to figure out which of the many things you did during that hour broke something.
When working with a growing and evolving system, the build-up of slow, full-system tests can become a problem gradually over a long time. It is often difficult to determine a single point when the tests became less useful, so developers should periodically take a step back to evaluate whether the test suite is still useful and instill the confidence it should be providing.
Automated tests also provide documentation. Code is read 10x more often than it is written. And well-written tests, like well-written code, give an insight into the test's intent.
We write tests to document the system, bridging the gap between the business stakeholders and the technical team by providing a common language. The language used to describe the tests and the system's expectations are in the terms of the business, and the tests execute that documentation to prove that the system fulfills the necessary conditions.
When a large team of developers and stakeholders work together to reach a shared understanding of the system’s behavior, ending up with a wide variety of descriptive styles can be hard to avoid. Major differences should be avoided, though, as reading the documentation of many styles is similar to reading a book with numerous authors — it results in a very jarring experience for readers.
To address this, we suggest auditing the readability of the tests at significant project milestones, such as when adding new members to the team or approaching a major release. This process will not only help get new people up to speed on system functionality but can also help ensure that documentation and desired functionality are maintained for future team members.
Depending on the type of test being written, some frameworks specifically call out sections as Given, When, or Then. Other frameworks do not make a specific differentiation between these sections, but the principles remain the same. Good tests should have a general structure that is easy to follow:
Given sections provide background detail and setup that is required before the test begins;
When sections give an action that is going to be tested and
Then, it provides assertions or expectations for the results of the actions provided.
Good test structure should make it fairly obvious which lines of test code belong to each section. For example, in general, Then often includes keywords such as “assert” or “expect.” When it becomes hard to distinguish each section from one another, or a significant amount of bouncing between Then and When exists, tests become significantly harder to maintain, and their purpose is often confusing. The context for any given test scenario should appear to be a concise setup of only a few lines, and the structure of actions and expectations should be clear to future readers and maintainers.
Expressive tests document the system by communicating the purpose of a piece of code, not just describing its functionality. For example, on a BlogPost object, you might have a test that checks for a published_at timestamp. An expressive test would check for this functionality and describe its purpose: “BlogPosts have a timestamp so that readers can discern how up-to-date the information is.” Or, to give a less trivial but also hypothetical example: “Patient data is encrypted in order to comply with HIPAA regulations.”
Expressive tests communicate purpose, not just functionality. They do it at the same level of abstraction as the unit under test. For example, a high-level UI integration test might be expressed as, “Logging in takes a user to the profile page.” If you're testing a more granular unit of the system, the test should still be expressive and purpose-driven: “ProfilePhotos only accept JPEGs and PNGs to simplify browser compatibility.”
It's easier to keep purpose in mind and write expressive tests if you identify the high-level acceptance criteria and tests first and then work your way down to the smallest unit under test. A team that writes consistently expressive tests and aligns with these principles communicates better — not only with each other but also with any developer who follows them in the codebase for the lifetime of the software product.
Our suite of automated tests allows us to work iteratively with confidence. This is because we have a safety net that ensures that we can confidently delete or add code because we know that we aren’t going to break something that already exists there. This means that knowledge from the developer and the team working on the project is initially encoded into the software asset so that they can focus on the part of the problem they are solving and make it easier to catch additional developers up on the project later.
It’s important to recognize that tests are code as well, and developers need to examine them to understand how they work. Write your tests with the same care as you do the application's actual features. If you do, you and any future developers who work on this project will thank you.
Like the code we write, we want our tests to embody intention-revealing names in the language of the business they are testing. We should look for opportunities to refactor and reuse tests as much as possible and then ensure that they are always run and correct by having continuous integration in code reviews that look at the tests as much as they do in the production code.
We write our tests first because we want to drive functionality from a failing test and get the benefits of TDD’s red-green-refactor cycle.
When writing our tests, we start from the outside and work our way in, taking the perspective of the software's end-user and working our way into the system's core business logic.
We follow three steps, and we do them iteratively:
Write new expectations the system does not fulfill. We create a test, and we watch it fail
Write code that makes the system meet that expectation, which causes the test to pass
Step back and consider the system as a whole. We look at our tradeoffs so far and ask ourselves if they are correctly tuned. Is this the best system that could meet the expectations that have now been written for it?
We can adjust the code if needed. Because we have the tests in place, we can adjust that code with confidence, and we don’t have to worry about regressing past behavior or failing to meet previously met expectations.
Michelangelo is quoted as saying that inside a block of marble is a sculpture waiting to be uncovered. The sculptor's job is to remove the excess and reveal what was already inside it. Similarly, for the skilled TDD practitioner, the software system is to be revealed by creating these tests and developing the system. Ultimately, using TDD lets us confidently create the best systems possible for customers that meet their needs today and tomorrow.
Software is full of theory. Theory can be helpful, but what counts for us is delivering value for customers. Testing the way we do lets that happen:
We can change systems confidently and quickly, knowing that our automated test safety net will flag any regressions
Our code is clear, even to people who haven’t touched it before (or in a while) because the tests show how to use it and interpret it
Feedback loops for us and for customers are fast, so we can respond to changes well, proactively fix any errors, and accommodate future growth effectively
The RoleModel Way of Testing helps us better manage the sustainable software assets we build with our customers and ensure we can deliver iterative value. We do this by ensuring our tests increase confidence and provide documentation.
RoleModel Software, Inc.
190 Rosewood Centre Drive, Suite 201
Holly Springs, NC 27540
