Running the Tests Locally
To run the tests, navigate to the root directory of the OpenLP project, and then run the following command:
On some platforms (like Ubuntu), you will need to run
On Windows you may need to run it using
pytest's file path e.g:
[OpenLP path]> C:\Python36\Scripts\pytest.exe
To run only a particular set of tests within a file, run the following command:
$ pytest tests/openlp_core/common/test_network_interfaces.py
Finally, to only run a particular test, run the following command:
$ pytest tests/openlp_core/common/test_network_interfaces.py::test_ip4_lo
Or, for a test method on a class:
$ pytest tests/openlp_core/common/test_network_interfaces.py::TestInterfaces::test_ip4_no_interfaces
How can you be sure you're writing the right tests? This is where code coverage comes in.
pytest supports the
coverage through a plugin, which can be installed on most systems.
On Ubuntu or Debian:
$ sudo apt install python3-pytest-cov
$ sudo dnf install python3-pytest-cov
Once you have coverage installed, just run the tests like this:
$ pytest --cov=openlp --cov-report=html
Then open up
index.html in the
htmlcov subdirectory in your current directory. The easiest way is to run a local webserver, like this:
$ (cd htmlcov && python3 -m http.server)
Then open your browser and navigate to http://localhost:8000 and you'll see something like this:
Logging into Jenkins
We use Jenkins for our Continuous Integration server. Jenkins checks out the code, runs all our tests automatically, and runs other code analysis tools like
flake8 on the code base. This gives us a set sane environment as a baseline for our tests.
- Go to our OpenLP Jenkins instance and create an account. Skip to step 3 if you've done this already.
- Once you've registered, contact Raoul (superfly in IRC) and give him your username. He will let you know once your account has the correct permissions set up.
- Log into Jenkins
- Click on the "Branch" tab
- Click on the play/time icon in the far right column of the first job in the list.
- Type in your branch on Launchpad's URL (lp:~username/openlp/branchname) and click "Build"
- Watch your branch being built
- If anything fails, you need to fix your code :-)
Running Jenkins Locally
Running the tests as described above is a little bit tedious, so we've written a script to make your life a little bit easier.
The script requires the python-jenkins module installed. This is already packaged in Debian and Fedora, but Windows and macOS developers will need to install it manually.
$ sudo apt install python3-jenkins
$ sudo dnf install python3-jenkins
$ pip install python-jenkins
Once you've pushed your code up to Launchpad, simply run the
jenkins_script.py script, and it will detect your remote branch, kick off the tests, and present you with the results on the command line. The script uses your username and either your password or a personal access token.
$ python3 scripts/jenkins_script.py -u <username> -p <password or token>
To make things slightly easier for yourself, below is a bash script which you can use to run the tests:
#!/bin/bash USER=<username> PASSWORD=<password> if [[ "$USER" == "" || "$PASSWORD" == "" ]]; then echo "No username or password set." exit 2 fi python3 scripts/jenkins_script.py -u $USER -p $PASSWORD $@
Save this script to somewhere in your
$PATH (I usually add
$HOME/bin to my
$PATH anyway, so I stuck it in there), and call it
ci. Then, when you want to run the tests all you have to do is navigate to your branch and run
Copy the result output into the description box of your merge proposal to show the team that you've run the tests on Jenkins.