tddscm.pl is a test-driven development tool for Scheme functions that take arguments and produce a return value. It works well for MIT Scheme programs that do not perform file I/O. tddscm.pl is written in Perl and is intended to work on most operating systems.

• Version 1.0 has been posted.

• You must have Perl installed on your machine (preferably v5.6 or above). If you do not have Perl, ActiveState provides a freely downloadable version that is highly recommended and easy to install/upgrade.

• Download the script, place it on your path, and rename it to tddscm.pl by replacing the .txt extension (needed for plain downloading on our server). For Windows NT/2000/XP users who have installed ActiveState perl, the .pl extension will already be associated with Perl and the script can be run as-is. Unix users should chmod +x the script (you can optionally rename it without the .pl extension if desired). Users of other operating systems may need to invoke Perl directly to use the script (e.g., perl tddpas.pl ...args... instead of simply tddpas.pl ...args...).

• Edit the script with a plain text editor to be sure that it reflects your Scheme installation correctly. In particular, check the definition of the variable $scheme_home in the User-configurable paths section near the top of the script. This path should reflect the path (including a trailing slash) where MIT Scheme is installed on your system. The value in the script by default is the default MIT Scheme installation directory under Windows 2000/XP. Unix users will definitely need to change this path.

The tddscm.pl script expects three command line arguments: the name of the Scheme source file containing the function to test, the name of the function to test within that file, and the name of a file of test cases. The test cases are plain text in a stylized format. Each test case is made up of two main parts: zero or more lines of input arguments (to be passed to the function as its arguments) and a corresponding set of zero or more lines reflecting the function's intended return result.

The test file format uses "//" at the beginning of a line to identify lines with special meaning to the test script. Specifically, any line that starts with the character sequence "//==" denotes the start of a test case. Any text on the remainder of the line serves as a "name" or label for the test case (for the purposes of identification if that test case fails). Lines following this marker are input lines. A later line starting with "//--" marks the end of the series of input lines and the start of the corresponding output lines. Any other line starting with "//" are treated as comments and are ignored by the test script.

For example, suppose you are writing a Scheme function that suggests spelling corrections. The function takes a single word as a character string as its argument, and returns as its result a list of zero or more strings representing possible spelling corrections. Here is a simple test case for such a function:

//== Testing misspelled word: seng
"seng"
//-- This is the expected output:
("sang" "sing" "song" "sung")

This single test case contains one set of input argument(s), followed by the desired function return value. A test case can have as many (or as few) input arguments as requested by the function, and these arguments can be spread over as many input lines as desired. Since any Scheme function produces only one return value, however, there should be exactly one line of output representing this return value.

In general, it is best to keep individual test cases as small and focused as possible--having lots of small test cases is preferred over having a few very large, complicated test cases. With a large test case, it is often hard to figure out exactly where or why a failure occurred. Further, you usually cannot run such large test cases until the entire program is complete--but smaller test cases can often be run sooner.

Your test case file can have as many test cases as you like. Just place them one after another. Be careful of blank lines, however--the testing script will treat them as significant lines in the output section of some test case. If you wish to separate your test cases within the file, use // comment lines instead (the test script will ignore them).

Suppose you are testing a function called spell-it in the source file spell-it.scm, with your test data stored in the file spell-tests.txt. You run the script like this:

    tddscm.pl spell-it.scm spell-it spell-tests.txt

The script runs tests using the following procedure:

• It parses the test case file, stripping out all the comments and splitting the remaining lines into a new Scheme source file that will load your source file, run each of the tests in turn, write the result of each test to a temporary output file, and then exit Scheme. This new Scheme file includes error handling to detect most kinds of errors and print them out instead of the desired function's output.

• It runs this new input file using Scheme.

• It walks through the actual output and the expected output files line by line, looking for mismatches. Leading space, trailing space, and upper case/lower case distinctions are ignored. Other sequences of white space are reduced down to single space characters, so differences in amounts of white space do not matter.

• It prints out the results, including a summary of the number of test cases run, the number that failed, and the number of runtime errors that occurred.

Output from a successful test run appears this way:

tddscm.pl v1.0   (c) 2003 Virginia Tech. All rights reserved.
Testing spell-it.scm/spell-it using spell-tests.txt

........................................

Tests Run: 40, Errors: 0, Failures: 0 (100.0%)

Suppose that the sample "seng" test case shown above is test case number 21 (test cases are counted from the beginning of the file starting at 1). Now suppose that we make a change to the program and as a result, the program produces "sunge" instead of "sung". Rerunning the tests produces the following:

tddscm.pl v1.0   (c) 2003 Virginia Tech. All rights reserved.
Testing spell-it.scm/spell-it using spell-tests.txt

....................F
case 21: FAILED: Testing misspelled word: seng
...................

Tests Run: 40, Errors: 0, Failures: 1 (97.5%)
Output has been saved in 1848.out.

If any output value produced by the program fails to match the corresponding expected output value contained in the test case, the entire test case will be considered a failure and will be identified as such (the label from the //== line will be used in the message). In addition, the temporary file containing the actual output of the function will be retained for your reference (it has a temporary name based on the process id).

The tddscm.pl script does match the produced output one-to-one with the expected output. Typically, however, any run-time error in your Scheme function will still produce a return result for that test case: an error message, including the cause of the error. As long as the output lines produced by the temporary Scheme input file match up correctly with your expected output, the test case location of all mismatches will be reported so you can track them down.

In practice, this is not usually a problem if you are following a TDD practice. That is because you will be writing test cases one at a time, and adding code a little at a time (just enough to implement the features of that new test case). That means in general, all of your test cases except the newest one will be working. If all of your test cases were working, and suddenly you get multiple failures in many test cases, then your latest modification introduced a bug that broke something. Fortunately, if you test often--every time you add a little bit of code--then you know exactly where the bug is without having to search for it. It has to be in the portion of the code you were just working in.

That is one big benefit of TDD. Being able to run the tests often, and doing it after each small piece of behavior you add gives you confidence in whether or not the code so far works correctly. Combine that with the practice of adding a test case for each and every capability before you write the code gives you a big leg up on completing a working solution.

Send any bugs or questions regarding tddscm.pl to Dr. Edwards.