Project 6 Tests

As mentioned in the handout, we are providing tests for this project. This page describes how to run the test script, what is being tested, and how to interpret the test output.

The test script is located at: ~cs3214/bin/server_unit_test.py The usage of the script is as such:

~cs3214/bin/server_unit_test.py -s server [-h] [-t testname] [-o outfile]

  -h      Show this help information
  -s server    File path to the server executeable
  -t testname    Run a single test
  -o outputfile    Send output from the server to an output file

Or simply server_unit_test.py -s ... if ~cs3214/bin/ is in your path.

• Basic, run all tests until a test suite fails

  ~cs3214/bin/server_unit_test.py -s ~/p6/server
  

• Running individual tests

  ~cs3214/bin/server_unit_test.py -s ~/p6/server -t test_method_check_1
  

• Send output to a file for a single test

  ~cs3214/bin/server_unit_test.py -s ~/p6/server -t test_method_check_1 -o ~/p6/output.txt
  

When you run the script, it will run either all tests or a single test. When running all of the tests, it will first run the minimum requirements tests. If those are passed, it runs the extra tests. If those are passed, it runs the malicious tests. When running a single test, the script only runs the single test.

It is HIGHLY RECOMMENDED that you send the output to a file. If the output from your server (anything you send to stdout) is not redirected, it will be displayed on the terminal inlined with the script's testing output. This may be very difficult to read, and discouraged for debugging.

The test script is broken up into three testing sections: Minimum Requirements (6 tests), Extra Tests (27 tests), and Malicious Tests (5 tests).

The minimum requirements tests must be passed to pass this class.

The extra tests are additional use cases for the server that should be taken into account, and will make up a significant portion of the grading. These requests are well formed, but otherwise invalid. For example, a request for GET /junk HTTP/1.1 should return a 404 status and possibly a 404 page.

The malicious tests are designed to attack your server and cause it to crash in various ways. For example, your server should not go down because it leaks file descriptors (fails to close them.)

If a test errors, you will get an output (for each test that errors) that looks like this:

FAIL:  Test Name: test_404_not_found_3
  Number Connections: 1
  Procedure:  Test a simple get request for an illegal object URL:
    GET /meminfo/junk HTTP/1.1

------------------------------------------------------------------------

TRACEBACK (most recent call last):
File "/home/courses/cs3214/bin/server_unit_test.py", line 490, in test_404_not_found_3
  server_check.run_404_check(self.http_connection, "/meminfo/junk")
File "/home/courses/cs3214/bin/server_check.py", line 92, in run_404_check
  assert server_response.status == httplib.NOT_FOUND, \
AssertionError:  Server failed to respond with a 404 status for obj=/meminfo/junk, gave response: 200

At the top is the test information. If you want to run this single test again, simple use the following flag: -t test_404_not_found_3. It includes the general procedure of what is being tested.

Below is the actual error that was generated. In this case, the provided URL should have created a 404 Not Found error, but /meminfo was returned instead with a 200 Ok message.

If you are having trouble testing your server with the script, I highly recommend the following commands as alternatives: curl, wget, and telnet. Use the -v flag with curl; examine the -S flag with wget. Telnet in particular will allow you to send a particular request to your server, and see exactly what is being sent back from your server; though it might not catch errors such as if the content-length information if your response doesn't match the actual length of what you mean to send back.

If there are any issues with the tests, PLEASE e-mail us at cs3214-staff@cs.vt.edu or post in the forum.