CUCC Expedition Handbook

Handbook Troggle - Automated Testing

Troggle Automated Testing

We have a suite of more than 70 smoke tests which are run manually by troggle programmers like this:

 troggle$ python manage.py test -v 1

These are 'end to end' tests which very quickly show whether something is badly broken. The tests are for two purposes only:

• To check whether anything has broken when we try a new version of python, Django or a Django plugin
• To check that the troggle system has been installed correctly on a new machine

This is surprisingly effective. Django produces excellently detailed tracebacks when an fault happens, which allow us to home in on the precise part of the code which has been broken by a version upgrade.

We do also have a handful of unit tests which just poke data into the database and check that it can be read out again.

The test code is all in troggle/core/TESTS/.

Example test

The test 'test_page_expofile' checks that a particular PDF is being served correctly by the web server and that the resulting page is the correct length of 2,299,270 bytes:

    def test_page_expofile(self):
        # Flat file tests.
        response = self.client.get('/expofiles/documents/surveying/tunnel-loefflerCP35-only.pdf')
        self.assertEqual(response.status_code, 200)
        self.assertEqual(len(response.content), 2299270)

Django test system

This test suite uses the the django test system. One of the things this does is to ensure that all the settings are imported correctly and makes it easy to specify a test as an input URL and expected HTML output using a Django object text client. It sets up a very fast in-memory sqlite database purely for tests. No tests are run with the real expo database.

Troggle tests

The tests can be run at a more verbose level by setting the -v 3 flag.

As yet we have no test database set up, so the in-memory database starts entirely empty. However we have 'fixtures' in troggle/core/fixtures/ which are JSON files containing dummy data which is read in before a few of the tests.

Automated testing on the server

Something is stopping the test suite running on the server. We haven't fixed this yet.

How you can help

We could do with a lot more unit tests which test small, specific things. If we have a lot of these it will make future re-engineering of troggle easier, as we can more confidently tackle big re-writes and still be sure that nothing is broken.

We haven't got any tests which check that the input parsers work. None.

Have a look at Wikpedia's review of types of software testing for ideas.

If you want to write some tests and are having trouble finding something which is untested, have a look at the list of url paths in the routing system in troggle/urls.py and look for types of url which do not appear in the test suite checks.