Test-driven development

From Wikipedia, the free encyclopedia

Jump to: navigation, search

Test-driven development (TDD) is a software development technique that uses short development iterations based on pre-written test cases that define desired improvements or new functions. Each iteration produces code necessary to pass that iteration's tests. Finally, the programmer or team refactors the code to accommodate changes. A key TDD concept is that preparing tests before coding facilitates rapid feedback changes. Note that test-driven development is a software design method, not merely a method of testing.

Test-Driven Development is related to the test-first programming concepts of Extreme Programming, begun in 1999,[1] but more recently is creating more general interest in its own right.[2]

Programmers also apply the concept to improving and debugging legacy code developed with older techniques.[3]

Software Testing portal

Contents

[edit] Requirements

Test-driven development requires developers to create automated unit tests that define code requirements before writing the code itself. The tests contain assertions that are either true or false. Running the tests rapidly confirms correct behavior as developers evolve and refactor the code. Developers use testing frameworks based on xUnit (see the list of unit testing frameworks for an exhaustive list) to create and automatically run sets of test cases.

[edit] Test-Driven Development Cycle

The following sequence is based on the book Test-Driven Development by Example,[4] which many consider to be the canonical source text on the concept in its modern form.

[edit] 1. Add a test

In test-driven development, each new feature begins with writing a test. This test must inevitably fail because it is written before the feature has been implemented. (If it does not fail, then the proposed “new” feature is obviated.) To write a test, the developer must clearly understand the feature's specification and requirements. The developer can accomplish this through use cases and user stories that cover the requirements and exception conditions. This could also imply a variant, or modification of an existing test. This is a differentiating feature of test-driven development versus writing unit tests after the code is written: it makes the developer focus on the requirements before writing the code, a subtle but important difference.

[edit] 2. Run all tests and see if the new one fails

This validates that the test harness is working correctly and that the new test does not mistakenly pass without requiring any new code.

The new test should also fail for the expected reason. This step tests the test itself, in the negative: it rules out the possibility that the new test will always pass, and therefore be worthless.

[edit] 3. Write some code

The next step is to write some code that will cause the test to pass. The new code written at this stage will not be perfect and may, for example, pass the test in an inelegant way. That is acceptable because later steps will improve and hone it.

It is important that the code written is only designed to pass the test; no further (and therefore untested) functionality should be predicted and 'allowed for' at any stage.

[edit] 4. Run the automated tests and see them succeed

If all test cases now pass, the programmer can be confident that the code meets all the tested requirements. This is a good point from which to begin the final step of the cycle.

[edit] 5. Refactor code

Now the code can be cleaned up as necessary. By re-running the test cases, the developer can be confident that refactoring is not damaging any existing functionality. The concept of removing duplication is an important aspect of any software design. In this case, however, it also applies to removing any duplication between the test code and the production code — for example magic numbers or strings that were repeated in both, in order to make the test pass in step 3.

[edit] Repeat

Starting with another new test, the cycle is then repeated to push forward the functionality. The size of the steps should always be small - as few as 1 to 10 edits between each test run. If new code does not rapidly satisfy a new test, or other tests fail unexpectedly, the programmer should undo or revert in preferrence to excessive debugging. Continuous Integration helps by providing revertable checkpoints. When using external libraries it is important not to make increments that are so small as to be effectively merely testing the library itself [2], unless there is some reason to believe that the library is buggy or is not sufficiently feature-complete to serve all the needs of the main program being written.

[edit] Development style

There are various aspects to using test-driven development, for example the principles of "Keep It Simple, Stupid" (KISS) and "You Ain't Gonna Need It" (YAGNI). By focusing on writing only the code necessary to pass tests, designs can be cleaner and clearer than is often achieved by other methods[4]. In Test-Driven Development by Example Kent Beck also suggests the principle "Fake it, till you make it".

To achieve some advanced design concept (such as a Design Pattern), tests are written that will generate that design. The code may remain simpler than the target pattern, but still pass all required tests. This can be unsettling at first but it allows the developer to focus only on what is important.

Write the tests first. The tests should be written before the functionality that is being tested. This has been claimed to have two benefits. It helps ensure that the application is written for testability, as the developers must consider how to test the application from the outset, rather than worrying about it later. It also ensures that tests for every feature will be written. When writing feature-first code, there is a tendency by developers and the development organisations to push the developer onto the next feature, neglecting testing entirely.

First fail the test cases. The idea is to ensure that the test really works and can catch an error. Once this is shown, the underlying functionality can be implemented. This has been coined the "Test-Driven Development Mantra", known as red/green/refactor where red means fail and green is pass.

Test-driven development constantly repeats the steps of adding test cases that fail, passing them, and refactoring. Receiving the expected test results at each stage reinforces the programmer's mental model of the code, boosts confidence and increases productivity.

Advanced practices of test-driven development can lead to Acceptance Test-driven development [ATDD] where the criteria specified by the customer are automated into acceptance tests, which then drive the traditional unit test-driven development [UTDD] process. This process ensures the customer has an automated mechanism to decide whether the software meets their requirements. With ATDD, the development team now has a specific target to satisfy, the acceptance tests, which keeps them continuously focused on what the customer really wants from that user story.

[edit] Benefits

A 2005 study found that using TDD meant writing more tests and, in turn, programmers that wrote more tests tended to be more productive.[5] Hypotheses relating to code quality and a more direct correlation between TDD and productivity were inconclusive.[6]

Programmers using pure TDD on new ("greenfield") projects report they only rarely feel the need to invoke a debugger. Used in conjunction with a version control system, when tests fail unexpectedly, reverting the code to the last version that passed all tests may often be more productive than debugging.[7][8]

Test-driven development may help to build software better and faster.[citation needed] It offers more than just simple validation of correctness, but can also drive the design of a program. By focusing on the test cases first, one must imagine how the functionality will be used by clients (in this case, the test cases). Therefore, the programmer is only concerned with the interface and not the implementation. This benefit is complementary to Design by Contract as it approaches code through test cases rather than through mathematical assertions or preconceptions.

The power test-driven development offers is the ability to take small steps when required. It allows a programmer to focus on the task at hand as the first goal is to make the test pass. Exceptional cases and error handling are not considered initially. Tests to create these extraneous circumstances are implemented separately. Another advantage is that test-driven development, when used properly, ensures that all written code is covered by a test. This can give the programmer, and subsequent users, a greater level of trust in the code.

While it is true that more code is required with TDD than without TDD because of the unit test code, total code implementation time is typically shorter.[9] Large numbers of tests help to limit the number of defects in the code. The early and frequent nature of the tests helps to catch defects early in the development cycle, preventing them from becoming endemic and expensive problems. Eliminating defects early in the process usually avoids lengthy and tedious debugging later in the project.

TDD can lead to more modularized, flexible, and extensible code. This effect often comes about because the methodology requires that the developers think of the software in terms of small units that can be written and tested independently and integrated together later. This leads to smaller, more focused classes, looser coupling, and cleaner interfaces. The use of the Mock Object design pattern also contributes to the overall modularization of the code because this pattern requires that the code be written so that modules can be switched easily between mock versions for unit testing or "real" version for deployment.

Because no more code is written than necessary to pass a failing test case, automated tests tend to cover every code path. For example, in order for a TDD developer to add an else branch off an existing if branch, the developer would first have to have written a failing test case that motivates the branch. As a result, the automated tests developed through strict application of TDD tend to be very robust, and can detect any significant functional mutation to the code base.

TDD encourages developers to put the minimum amount of functional code into amodules and maximise the logic that is extracted into testable library code, using fakes and mocks to represent the outside world.

[edit] Limitations

There are situations where the TDD approach is difficult to use, or where it increases costs without having appropriate benefits. Examples are graphical interfaces that need a full functional test, projects that use external staff that may not have deep-enough knowledge about the project functionality, writing tests that fit the code that is written immediately afterwards, but that do not comply with the functional specification.

To make TDD work, the following points should be taken into consideration:

  1. In situations that need full functional tests.
  2. The entire organization should believe that Test-Driven Development is going to improve the product. Otherwise, the management will feel that time spent writing tests is wasted time [1].
  3. Tests themselves should not become part of the maintenance overhead of a project. Badly written tests, for example ones that check hard-coded error strings or that are themselves prone to failure, are expensive to maintain. There is a risk that tests that regularly generate false failures will be ignored, so that when a real failure occurs it may not be detected. It is possible to write tests for low and easy maintenance, for example by reusing error strings, and this should be a goal during the 'Refactor' phase described above.
  4. As unit tests are typically created by the developer who will also write the code that is being tested, they may share the same blind spots with the code: If e. g. a developer does not realize that certain input parameters to a function must be checked, most likely neither the test nor the code will verify these input parameters. If the developer misinterprets the requirements specification for the unit being developed, both the tests and the code will be wrong.
  5. If unit tests are created by developers, the high number of passing unit tests may bring a false sense of security, resulting in less additional QA activities, such as integration testing and compliance testing.

[edit] Code Visibility

Test-suite code clearly has to be able to access the code it is testing. On the other hand normal design criteria such as information hiding, encapsulation and the separation of concerns should not be compromised. Therefore unit test code for TDD is usually written within the same project or module as the code being tested.

In object oriented design this still does not provide access to private data and methods. Therefore, extra work must be done to create unit tests. In Java, for example, a developer can use reflection to access fields that are marked private[10].

[edit] Fakes, mocks and integration tests

Unit tests are so named because they each test one unit of code. Whether a module of code has hundreds of unit tests or only five is irrelevant. A test suite should never cross process boundaries in a program, let alone network connections. Doing so introduces delays that make tests run slowly and discourage developers from running the whole suite. Introducing dependencies on external modules or data also turns unit tests into integration tests. If one module misbehaves in a chain of interrelated modules, it may not be clear where to look for the cause of the failure.

When code under development relies on a database, a Web service, or any other external process or service, enforcing a unit-testable separation is an opportunity and a driving force to design more modular, more testable and more reusable code.[11] Two steps are necessary:

  1. Whenever external access is going to be needed in the final design, an interface should be defined that describes the access that will be available.
  2. The interface should be implemented in two ways, one of which really accesses the external process, and the other of which is a fake or mock object. Fake objects need do little more than add a message such as “Person object saved” to a trace log or to the console. Mock objects differ in that they themselves contain test assertions that can make the test fail, for example, if the person's name and other data are inconsistent. Fake and mock object methods that return data, ostensibly from a data store or user, can help the test process by always returning the same, realistic data that tests can rely upon. They can also be set into predefined fault modes so that error-handling routines can be developed and reliably tested.

A corollary of this approach is that the actual database or other external-access code is never tested by the TDD process itself. To avoid this, other tests are needed that instantiate the test-driven code with the “real” implementations of the interfaces discussed above. Many developers find it useful to keep these tests quite separate from the TDD unit tests, and refer to them as integration tests. There will be fewer of them, and they need to be run less often than the unit tests. They can nonetheless be implemented using the same testing framework, such as xUnit.

Integration tests that alter any persistent store or database should always be careful to consider the initial and final state of the files or database, even if any test fails. This can be achieved using some combination of the following techniques where relevant:

  • The TearDown method integrated into many test frameworks.
  • try...catch...finally exception handling structures where available.
  • Database transactions where a transaction atomically includes perhaps a write, a read and a matching delete operation.
  • Taking a “snapshot” of the database before running any tests and rolling back to the snapshot after each test run. This may be automated using a framework such as Ant or NAnt.
  • Initialising the database to a clean state before tests, rather than cleaning up after them. This may be relevant where cleaning up may make it difficult to diagnose test failures by deleting the final state of the database before detailed diagnosis can be performed.

Frameworks such as jMock, NMock, EasyMock, Typemock, jMockit, PowerMock or Rhino Mocks exist to make the process of creating and using complex mock objects easier.

[edit] References

  1. ^ "Extreme Programming", Computerworld (online), December 2001, webpage: Computerworld-appdev-92.
  2. ^ a b Newkirk, JW and Vorontsov, AA. Test-Driven Development in Microsoft .NET, Microsoft Press, 2004.
  3. ^ Feathers, M. Working Effectively with Legacy Code, Prentice Hall, 2004
  4. ^ a b Beck, K. Test-Driven Development by Example, Addison Wesley, 2003
  5. ^ Erdogmus, Hakan; Morisio, Torchiano. "On the Effectiveness of Test-first Approach to Programming". Proceedings of the IEEE Transactions on Software Engineering, 31(1). January 2005. (NRC 47445). http://iit-iti.nrc-cnrc.gc.ca/publications/nrc-47445_e.html. Retrieved on 2008-01-14. "We found that test-first students on average wrote more tests and, in turn, students who wrote more tests tended to be more productive." 
  6. ^ Proffitt, Jacob. "TDD Proven Effective! Or is it?". http://theruntime.com/blogs/jacob/archive/2008/01/22/tdd-proven-effective-or-is-it.aspx. Retrieved on 2008-02-21. "So TDD's relationship to quality is problematic at best. Its relationship to productivity is more interesting. I hope there's a follow-up study because the productivity numbers simply don't add up very well to me. There is an undeniable correlation between productivity and the number of tests, but that correlation is actually stronger in the non-TDD group (which had a single outlier compared to roughly half of the TDD group being outside the 95% band)." 
  7. ^ Clark, Mike. "Test-Driven Development with JUnit Workshop". Clarkware Consulting, Inc.. http://clarkware.com/courses/TDDWithJUnit.html. Retrieved on 2007-11-01. "In fact, test-driven development actually helps you meet your deadlines by eliminating debugging time, minimizing design speculation and re-work, and reducing the cost and fear of changing working code." 
  8. ^ Llopis, Noel (20 February 2005). "Stepping Through the Looking Glass: Test-Driven Game Development (Part 1)". Games from Within. http://www.gamesfromwithin.com/articles/0502/000073.html. Retrieved on 2007-11-01. "Comparing [TDD] to the non-test-driven development approach, you're replacing all the mental checking and debugger stepping with code that verifies that your program does exactly what you intended it to do." 
  9. ^ Müller, Matthias M.; Padberg, Frank. "About the Return on Investment of Test-Driven Development" (PDF). Universität Karlsruhe, Germany. 6. http://www.ipd.uka.de/mitarbeiter/muellerm/publications/edser03.pdf. Retrieved on 2007-11-01. 
  10. ^ http://www.onjava.com/pub/a/onjava/2003/11/12/reflection.html
  11. ^ Fowler, Martin (1999). Refactoring - Improving the design of existing code. Boston: Addison Wesley Longman, Inc.. ISBN 0-201-48567-2. 

[edit] See also

[edit] External links

[edit] Testing-Driven Development X Design by Contract

[edit] Java Testing

[edit] Windows/.NET Testing

[edit] Testing in other languages

Retrieved from "http://en.wikipedia.org/wiki/Test-driven_development"
Personal tools