In Agile, we believe in the shift left testing approach to software and system testing. This means testing is performed earlier in the lifecycle (i.e. moved left on the delivery timeline). It is the first half of the maxim "Test early and often." The continuous delivery method provides many new challenges for test engineers. With the iterative development approach, it’s important to maintain the quality of the product in every step of the delivery cycle. This approach is often called continuous testing - this means testing from the very first iteration, reducing bottlenecks and handover times by combining various tools and practices. It is also ones associated so far with different roles in the engineering process. Testing in continuous delivery means building and maintaining a quality assurance strategy in the whole development and delivery cycle.

 

Test-driven development (TDD) is an advanced technique of using automated unit tests (referenced in the above Pyramid) to drive the design of software and force decoupling of dependencies. The result of using this practice is a comprehensive suite of unit tests that can be run at any time to provide feedback that the software is still working. This technique is heavily emphasized by those using Agile development methodologies. The concept is to “get something working now and perfect it later.” After each test, refactoring is done and then the same or a similar test is performed again. The process is iterated as many times as necessary until each unit is functioning according to the desired specifications.

Characteristics of a Good Unit Test

A good unit test has the following characteristics.

  • Runs fast, runs fast, runs fast. If the tests are slow, they will not be run often.
  • Separates or simulates environmental dependencies such as databases, file systems, networks, queues, and so on. Tests that exercise these will not run fast, and a failure does not give meaningful feedback about what the problem actually is.
  • Is very limited in scope. If the test fails, it's obvious where to look for the problem. Use few Assert calls so that the offending code is obvious. It's important to only test one thing in a single test.
  • Runs and passes in isolation. If the tests require special environmental setup or fail unexpectedly, then they are not good unit tests. Change them for simplicity and reliability. Tests should run and pass on any machine. The "works on my box" excuse doesn't work.
  • Often uses stubs and mock objects. If the code being tested typically calls out to a database or file system, these dependencies must be simulated, or mocked. These dependencies will ordinarily be abstracted away by using interfaces.
  • Clearly reveals its intention. Another developer can look at the test and understand what is expected of the production code.

Benefits of ATDD & TDD

  • The suite of unit tests provides constant feedback that each component is still working.
  • The unit tests act as documentation that cannot go out-of-date, unlike separate documentation, which can and frequently does.
  • When the test passes and the production code is refactored to remove duplication, it is clear that the code is finished, and the developer can move on to a new test.
  • Test-driven development forces critical analysis and design because the developer cannot create the production code without truly understanding what the desired result should be and how to test it.
  • The software tends to be better designed, that is, loosely coupled and easily maintainable, because the developer is free to make design decisions and refactor at any time with confidence that the software is still working. This confidence is gained by running the tests. The need for a design pattern may emerge, and the code can be changed at that time.
  • The test suite acts as a regression safety net on bugs: If a bug is found, the developer should create a test to reveal the bug and then modify the production code so that the bug goes away and all other tests still pass. On each successive test run, all previous bug fixes are verified.
  • Reduced debugging time!

Behavior-Driven Development (BDD) combines the general techniques and principles of TDD with ideas from domain-driven design. BDD is a design activity where you build pieces of functionality incrementally guided by the expected behavior. The focus of BDD is the language and interactions used in the process of software development. Behavior-driven developers use their native language in combination with the language of Domain Driven Design to describe the purpose and benefit of their code.


A team using BDD should be able to provide a significant portion of “functional documentation” in the form of User Stories augmented with executable scenarios or examples. BDD is usually done in very English-like language helps the Domain experts to understand the implementation rather than exposing the code level tests. It’s usually defined in a GWT format: GIVEN WHEN & THEN.

  1. BDD tests are written in an English-like language.
  2. BDD focuses on the behavioral aspect of the system.
  3. BDD is customer-focused.
  4. BDD gives a clearer understanding as to what the system should do from the perspective of the developer and the customer.
  5. BDD is a way to ensure consistency between requirements and the developer tests.

User Acceptance Testing (UAT) involves validating software in a real setting by the intended audience. The aim is not so much to check the defined requirements but to ensure that the software satisfies the customer’s needs. Agile methodologies put stringent demands on UAT, if only for the frequency at which it needs to be conducted due to the iterative development of small product releases. In this setting, traditional in-person meetings might not scale up well. Complementary ways are needed to reduce the costs of developer-customer collaboration during UAT. This work introduces a wiki-based approach where customers and developers asynchronously collaborate: developers set the UAT scaffolding that will later shepherd customers when testing. To facilitate understanding, mind maps are used to represent UAT sessions. To facilitate engagement, a popular mind map editor, FreeMind, is turned into an editor for FitNesse, the wiki engine in which these ideas are borne out. The approach is evaluated through a case study involving three real customers. First evaluations are promising. Though at different levels of completeness, the three customers were able to complete a UAT. Customers valued asynchronicity, mind map structuredness, and the transparent generation of documentation out of the UAT session.