TestRunner Overview
TestRunner is a portable, easy to use, automated software testing utilty. Designed with command line utilities in mind, TestRunner supports exit code monitoring, output file consistency checking, and time-based performance testing. Installation and setup is simple and can be completed in minutes. Adding tests only requires that a new test directory be created with the necessary configuration files for the application. On demand expected results generation takes care of the hard work, completing test creation automatically.
Featuring built-in (subversion) repository operations, TestRunner streamlines collaborative development. Master and slave repository modes, in addition to the default local mode, facilitate integration with automated builder systems, such as BuildBot.
Benefits of Software Testing
Software testing allows developers to continually monitor application functionality, while moving forward with development. When integrated with an automated builder system, testing can quickly catch functional and performance regressions, providing instant feedback and ensuring continual quality improvement.
The TestRunner Approach
Many approaches to software testing exist, each with its own strengths and weaknesses. TestRunner has been designed to test a complete application from the user's perspective, running the application with specific configuration settings and verifying that expected output has been generated.
Benefits
- Ensures consistent results for a given configuration
- Upholds end user expectations
- Verifies successful component integration
- Single tests can supply large code coverage
- Completely implementation agnositic
- Lightweight, requiring minimal setup and maintenance
Weaknesses
- Large code coverage of tests may make it difficult to identify cause of failure
- Can only cover code exposed through user interfaces
- Requires completed functionality for automated expected results
- Currently has no capability to test interactive (especially graphical) user interfaces
Projects using TestRunner
License, Feedback and Suggestions
TestRunner has been licensed under a BSD-style open source license that is OSI-approved. You are welcome to use the code however you see fit, following the simple conditions of the license. As stated in the license, you are not required to release any modifications to TestRunner. However, if you make improvements that you think should be included in future releases, please feel free to send them to me [contact].
Along similar lines, general feedback and suggestions are strongly encouraged. If you decide to utilize TestRunner in your project (both unmodified and modified versions), I would love hear about it. I will include a link to your project above, if desired.
Thanks for considering TestRunner!
Last Updated: 2007-04-11
