Testing

OGC CITE

Compliance benchmarking is done via the OGC Compliance & Interoperability Testing & Evaluation Initiative. The pycsw wiki documents testing procedures and status.

Tester

The pycsw tester framework (in tester) is a collection of testsuites to perform automated regession testing of the codebase.

Running

The tester framework can be run from tester:

$ cd /path/to/pycsw
$ cd tester
$ python ./run_tests.py
# lots of output

The tester runs HTTP GET and POST requests. The expected output for each test can be found in expected. Results are categorized as passed, failed, or initialized. A summary of results is output at the end of the run.

Failed Tests

If a given test has failed, the output is saved in results. The resulting failure can be analyzed by running diff expected/name_of_test.xml results/name_of_test.xml to find variances.

Test Suites

The tester framework is run against a series of ‘suites’ (in tester/suites), each of which specifies a given configuration to test various functionality of the codebase. Each suite is structured as follows:

• tester/suite/default.cfg: the configuration for the suite
• tester/suite/post: directory of XML documents for HTTP POST requests
• tester/suite/get/requests.txt: directory and text file of KVP for HTTP GET requests

When the tester is invoked, the following operations are run:

• pycsw configuration is set to tester/suite/default.cfg
• HTTP POST requests are run against tester/suite/post/*.xml
• HTTP GET requests are run against each request in tester/suite/get/requests.txt

The CSV format of tester/suite/get/requests.txt is testname,request, with one line for each test. The testname value is a unique test name (this value sets the name of the output file in the test results). The request value is the HTTP GET request. The PYCSW_SERVER token is replaced at runtime with the URL to the pycsw install.

Adding New Tests

To add tests to an existing suite:

• for HTTP POST tests, add XML documents to tester/suite/post
• for HTTP GET tests, add tests (one per line) to tester/suite/get/requests.txt
• run python ./run_tests.py <url>

To add a new test suite:

• create a new directory under tester/suites (e.g. foo)
• create a new configuration in tester/suites/foo/default.cfg. Ensure that all file paths are relative to path/to/pycsw
• populate HTTP POST requests in tester/suites/foo/post
• populate HTTP GET requests in tester/suites/foo/get/requests.txt
• run python ./run_tests.py <url>

Web Testing

You can also use the pycsw tester via your web browser to perform sample requests against your pycsw install. The tester is located in tester/. To generate the HTML page, run gen_html.py:

$ python ./gen_html.py > index.html

Then navigate to http://host/path/to/pycsw/tester/index.html.