Test automation
From Wikipedia, the free encyclopedia
 |
This article includes a list of references or external links, but its sources remain unclear because it lacks inline citations. Please improve this article by introducing more precise citations where appropriate. (February 2009) |
- Compare with Manual testing.
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.
Contents |
[edit] Overview
Although manual exploratory testing can find many defects in a software application, it is a laborious and time consuming process. In addition it may not be effective in finding certain classes of defects. Test automation is a process of writing a computer program to do testing that would otherwise need to be done manually. Once the testing has been automated, a large number of test cases can be validated quickly. This is most cost effective for software products that will have a long shelf life, because even minor patches over the lifetime of the application can cause features to break which were working at an earlier point in time.
There are two ways to design the tests:
- Black box testing. The test developer has no knowledge of the inner workings of the program. The tests cover all the cases that an end user would run into. The completeness of the tests depends on the test developer's expertise using the application.
- White box testing. The test developer has full knowledge of the inner workings of the program. The tests ensure each pathway through the source code has been exercised and is working properly. Its completeness can be measured by Code coverage metrics.
There are two general approaches to test automation:
- Graphical user interface testing. A testing framework generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.
- Code-driven testing. The public (usually) interface to classes, modules, or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.
Test automation can be expensive, and it is usually employed in combination with manual exploratory testing. It can be made cost-effective in the longer term though, especially in regression testing. One way to generate test cases automatically is model-based testing where a model of the system is used for test case generation, but research continues into a variety of methodologies for doing so.
What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team has to take. Selecting the correct features of the product for automation largely decides the success of the automation. Unstable features or the features which are undergoing changes should be avoided.
[edit] Graphical User Interface (GUI) testing
Many test automation tools provide record and playback features that allow users to record interactively user actions and replay it back any number of times, comparing actual results to those expected. The advantage of this approach is that it requires little or no software development. This approach can be applied to any application that has a graphical user interface. However, reliance on these features poses major reliability and maintainability problems. Relabelling a button or moving it to another part of the window may require the test to be re-recorded.
A variation on this type of tool is for testing of web sites. Here, the "interface" is the web page. This type of tool also requires little or no software development. However, such a framework utilizes entirely different techniques because it is reading html instead of observing window events.
[edit] Code-driven testing
A growing trend in software development is to use testing frameworks such as the xUnit frameworks (for example, JUnit and NUnit) which allow the code to conduct unit tests to determine whether various sections of the code are acting as expected under various circumstances. Test cases describe tests that need to be run on the program to verify that the program runs as expected.
Code driven test automation is a key feature of Agile software development. Proponents of this methodology argue that it produces software that is both more reliable and less expensive than code that is tested by manual exploration. It is considered more reliable because the code coverage is better, and because it is run constantly during development rather than once at the end of a waterfall development cycle. And it is less expensive because the developer discovers the defect immediately upon making a change, when it is least expensive to fix.
[edit] What to test
Testing tools can help automate tasks such as product installation, test data creation, GUI interaction, problem detection (consider parsing or polling agents equipped with oracles), defect logging, etc., without necessarily automating tests in an end-to-end fashion.
One must keep following points when thinking of test automation:
- Platform and OS independence
- Data driven capability (Input Data, Output Data, Meta Data)
- Customizable Reporting (DB Access, crystal reports)
- Email Notifications (Automated notification on failure or threshold levels)
- Easy debugging and logging
- Version control friendly – minimum or zero binary files
- Extensible & Customizable (Open APIs to be able to integrate with other tools)
- Common Driver (Ant or Maven)
- Headless execution for unattended runs (For integration with build process or batch runs)
- Support distributed execution environment (distributed test bed)
- Distributed application support (distributed SUT)
[edit] Framework approach in automation
A framework is an integrated system that sets the rules of Automation of a specific product. This system integrates the function libraries, test data sources, object details and various reusable modules. These components act as small building blocks which need to be assembled in a regular fashion to represent a business process. Thus, framework provides the basis of test automation and hence simplifying the automation effort.
There are various types of frameworks. They are categorized on the basis of the automation component they leverage. These are:
1. Data-driven testing
2. Modularity-driven testing
3. Keyword-driven testing
4. Hybrid testing
5. Model-based testing
[edit] See also
- Software testing
- Test automation framework
- Unit test
- Xnee recorder and replayer for X Window System
[edit] References
- Elfriede Dustin, et al: Automated Software Testing. Addison Wesley, 1999, ISBN 0-20143-287-0
- Mark Fewster & Dorothy Graham (1999). Software Test Automation. ACM Press/Addison-Wesley. ISBN 978-0201331400.
- Roman Savenkov: How to Become a Software Tester. Roman Savenkov Consulting, 2008, ISBN 978-0-615-23372-7
- Hong Zhu et al. (2008). AST '08: Proceedings of the 3rd International Workshop on Automation of Software Test. ACM Press. ISBN 978-1-60558-030-2.
