Development:Unit Tests

Introduction
If you haven't read through Test-Driven Development yet, go and read that first. That will explain some of the reasoning behind writing tests.

A very basic test case (or test suite as it is sometimes called), with two tests:


 * 1) We have a class which holds a set of tests related to each other.
 * 2) Each method in the class is a test (you can see, it ends with "_test").
 * 3) In order for a method to be recognised as a test, it needs to end with "_test".
 * 4) Each test is divided into 3 sections: GIVEN, WHEN, THEN:
 * 5) * GIVEN: These are the things we start with
 * 6) * WHEN: This is what happens
 * 7) * THEN: These are the results
 * 8) For simplicity's sake I have two tests which each have 1 GIVEN-WHEN-THEN combination. As a general rule, if you have more than one WHEN, then you probably need to split your test into smaller tests. In some cases this is not possible, but this is a good rule of thumb.

I've called this class, however it will usually have a name related to the module being tested.

To actually run the tests I am using a python test running framework called nosetests (or just nose). You'll see that in the tests I use the assert-methods of the TestCase-class to determine that my results are what I expect.

nose will catch the AssertionError exceptions that these assert-methods raise if the statement is not as expected, and will record that test as having failed. The failure-message is an optional argument of the assert-methods, if none is provided a default is used.

You'll see this test is passing a boolean into a function that looks like it should take a string. Tests should check to see how a piece of code behaves depending on what input it receives.

In this function's case, it is able to take a boolean so I test that it can take a boolean value and will return the correct result. Here is another test for the same function: This one tests what happens if I send in invalid input.

Mocked Functions
Right, so now we have this other function, and it calls some other function - how do I know that I'm testing just this function and not the other function I'm calling?

Quite simply, we use a fantastic library called "mock" which is able to replace functions, methods and classes with our own magic versions that don't do anything.

Using MagicMock
One of the classes that can be used to replace other classes and functions is the MagicMock class.

The mocked function pretends it is the function we are calling, returns a fixed value and keeps track of what it was called with so it can be checked later.

The translate function above is unusual in that the function called is passed in. In most cases this doesn't have. This is where the "patch" method comes in.

Using The patch Function
The patch method patches an object and inserts itself into the place of the object you want to mock out.

Here is another test, this one uses the patch function

So what I'm doing is I'm actually substituting os.path.exists and os.makedirs for my own mock methods, and then when I call the function it calls my mocks instead of the real functions.

This way we can completely control all the data around a function and check that it reacts accordingly. These are proper unit tests, they test the smallest possible unit (or block) of code.

We won't always be able to do this level of testing, but this is a good way to get used to writing tests

Notice when you patch a method, you need to patch it where it is imported. So you don't just patch "os.path.exists", you need to patch "openlp.core.lib.os.path.exists".

Writing Your Tests
When writing a set of tests, the easiest way to do it is to start with the shortest path that the code takes.

For instance, let's look at this function:

The shortest path through this function would happen if os.path.exists returned False. So let's write a test which mocks out os.path.exists and makes it return False.

Triggering and Handling Exceptions
Sometimes you need to check that a function handles exceptions properly. To do this you need to both raise an exception and then check that the exception was handled correctly in the code. Once again, the mock library comes to our rescue.

To raise an exception, simply mock out the method that should raise the exception, and then add a *side effect* to the method. If the function you're testing is supposed to raise an exception itself, or allow one to bubble up, then you can assert that the exception is raised using the self.assertRaises method from the TestCase class.

Write Thorough Tests

 * 1) Check what you got back from the function and that it was what you expected.
 * 2) Test each code path, start with the shortest.
 * 3) Check the mocked functions that you expected to be called were called.
 * 4) Check they were called with the expected parameters.

An example of writing a test for the get_filesystem_encoding function in openlp/core/utils/__init__.py

Location of tests
Tests are located under the  directory. Inside this is a  directory for the unit tests, which contain additional directories for each namespace.

Software required
In order to run the unit tests, you will need the following Python packages/libraries installed:
 * 1) Mock
 * 2) Nose

On Ubuntu you can simple install the python3-mock and python3-nose packages. Most other distributions will also have these packages. On Windows and Mac OS X you will need to use pip or easy_install to install these packages.

Running the tests
See the Running Tests page on how to run the tests you've written.