tddpas.pl is a test-driven development tool for programs that are directly executable on the program and that use standard input and standard output for I/O. It works well for FreePascal programs. tddpas.pl is written in Perl and is intended to work on most operating systems.

• Version 1.3 has been posted. Now shows both expected and received output for each failed test case.

• Version 1.2 has been posted. Some minor internal changes were made, and the "..." output for passed tests now wraps at 78 columns.

• Version 1.1 has been posted. The only change is that now, if you do not place a label on the //== line of a test case, the first line of input for that test case will be used as the label instead.

• You must have Perl installed on your machine (preferably v5.6 or above). If you do not have Perl, ActiveState provides a freely downloadable version that is highly recommended and easy to install/upgrade (the requested contact information is not required).

• Download the script, place it on your path, and rename it to tddpas.pl by replacing the .txt extension (needed for plain downloading on our server). For Windows NT/2000/XP users who have installed ActiveState perl, the .pl extension will already be associated with Perl and the script can be run as-is. Unix users should chmod +x the script (you can optionally rename it without the .pl extension as well if you like). Users of other operating systems may need to invoke Perl directly to use the script (e.g., perl tddpas.pl ...args... instead of simply tddpas.pl ...args...).

The tddpas.pl script expects two command line arguments: the name of the executable program to test and the name of a file of test cases. The test cases are plain text in a stylized format. Each test case is made up of two main parts: zero or more lines of input (to be passed to the program on standard input) and a corresponding set of zero or more lines of output (that should be produced in response on standard output).

The test file format uses "//" at the beginning of a line to identify lines with special meaning to the test script. Specifically, any line that starts with the character sequence "//==" denotes the start of a test case. Any text on the remainder of the line serves as a "name" or label for the test case (for the purposes of identification if that test case fails). Lines following this marker are input lines. A later line starting with "//--" marks the end of the series of input lines and the start of the corresponding output lines. Any other line starting with "//" are treated as comments and are ignored by the test script.

For example, suppose you are writing a program that suggests spelling corrections. The program reads one word on a line by itself, prints out one or more possible spelling corrections, and repeats until it runs out of input. Here is a simple test case for such a program:

//== Testing misspelled word: seng
seng
//-- This is the expected output:
sang
send
sent
sing
song
sung

This single test case contains one line of input, followed by six lines of output. A test case can have as many (or as few) lines of input, and as many (or as few) lines of output as you would like. In general, however, it is best to keep individual test cases as small and focused as possible--having lots of small test cases is preferred over having a few very large, complicated test cases. With a large test case, it is often hard to figure out exactly where or why a failure occurred.

Your test case file can have as many test cases as you like. Just place them one after another. Be careful of blank lines, however--the testing script will treat them as significant lines in the input or output section of some test case. If you wish to separate your test cases within the file, use // comment lines instead, since they will be ignored by the script.

Suppose you are testing a program called spell-it with your test data stored in the file spell-tests.txt. You run the script like this:

    tddpas.pl spell-it spell-tests.txt

The script runs tests using the following procedure:

• It parses the test case file, stripping out all the comments and splitting the remaining lines into a file of input lines and a file of expected output lines.

• It runs your program, piping in the file of input lines and collecting the output in a third file.

• It walks through the actual output and the expected output files line by line, looking for mismatches. Leading space, trailing space, and upper case/lower case distinctions are ignored. Other sequences of white space are reduced down to single space characters, so differences in amounts of white space do not matter.

• It prints out the results, including a summary of the number of test cases run, the number that failed, and the number of runtime errors that occurred.

Output from a successful test run appears this way:

tddpas.pl v1.2   (c) 2003 Virginia Tech. All rights reserved.
Testing spell-it using spell-tests.txt

........................................

Tests Run: 40, Errors: 0, Failures: 0 (100.0%)

Suppose that the sample "seng" test case shown above is test case number 21 (test cases are counted from the beginning of the file starting at 1). Now suppose that we make a change to the program and as a result, the program produces "sunge" instead of "sung". Rerunning the tests produces the following:

tddpas.pl v1.2   (c) 2003 Virginia Tech. All rights reserved.
Testing spell-it using spell-tests.txt

....................F
case 21: FAILED: Testing misspelled word: seng
...................

Tests Run: 40, Errors: 0, Failures: 1 (97.5%)
Output has been saved in 1848.out.

If any output line produced by the program fails to match the corresponding expected output line contained in the test case, the entire test case will be considered a failure and will be identified as such (the label from the //== line will be used in the message). In addition, the temporary file containing the actual output of the program will be retained for your reference (it has a temporary name based on the process id).

The tddpas.pl script does match the produced output one-to-one with the expected output. As a result, if the program fails to produce one line (or produces an extra line), all remaining test cases will fail because the remaining lines will be off by one during the comparison. Fortunately, the script identifies the specific test case where the first problem was detected. As long as lines match up correctly, the test case location of all mismatches will be reported so you can track them down.

In practice, this is not usually a problem if you are following a TDD practice. That is because you will be writing test cases one at a time, and adding code a little at a time (just enough to implement the features of that new test case). That means in general, all of your test cases except the newest one will be working. If all of your test cases were working, and suddenly you get multiple failures in many test cases, then your latest modification introduced a bug that broke something. Fortunately, if you test often--every time you add a little bit of code--then you know exactly where the bug is without having to search for it. It has to be in the portion of the code you were just working in.

That is one big benefit of TDD. Being able to run the tests often, and doing it after each small piece of behavior you add gives you confidence in whether or not the code so far works correctly. Combine that with the practice of adding a test case for each and every capability before you write the code gives you a big leg up on completing a working solution.

Send any bugs or questions regarding tddpas.pl to Dr. Edwards.