OpenFisca has three sorts of tests:

  • unit tests
  • test-case tests
  • scenario tests

Run tests

OpenFisca uses nose to run its unit tests. Here are some useful commands.

  • Run the whole test suite:
      make test
    which is available at least in Core, France and Web-API repositories.
  • Run a specific test:
      nosetests openfisca_france/tests/test_parameters.py
  • Hide log of failing test:
      nosetests --nologcapture openfisca_france/tests/test_parameters.py
  • Display log of successful test:
      nosetests --debug=openfisca_core openfisca_france/tests/test_parameters.py

YAML tests

Formulas are tested with YAML tests.

ipdb debugger

If a test fails, you can execute it with the debug nose plugin:

nosetests --pdb openfisca_core/tests/test_tax_scales.py

You'll be dropped in the pdf debugger shell when an error occurs.

You can specify the exact test to launch:

nosetests --pdb openfisca_core/tests/test_tax_scales.py:test_linear_average_rate_tax_scale

The nose-ipdb plugin is more user-friendly (because it uses the ipdb debugger instead of pdb). In this case, just use the --ipdb option rather than --pdb. See also the --ipdb-failure option.

In case you want to set a breakpoint manually, in order to enter the debugger shell before an errors occurs, copy-paste this line in your code:

import nose.tools; nose.tools.set_trace(); import ipdb; ipdb.set_trace()

This needs ipdb to be installed.

Hint: use the snippets feature of your favorite text editor to save this line, for example give it the name "breakpoint".

Travis automated tests

OpenFisca uses Travis CI to run tests automatically after each git push.

The repositories tested by Travis are:

The OpenFisca website hosts a summary page of the build statuses: https://www.openfisca.fr/build-status

Travis tests other git branches than master too. For example: OpenFisca-Core.

For OpenFisca-France, when testing a branch, if there is a branch in OpenFisca-Core with the same name, Travis will checkout it before running the tests. This is done by this script.

Idem for OpenFisca-Web-API with this script which is sightly different because it handles more dependencies.

Ludwig tests

To download tests from Ludwig (the tests tool from Mes aides), see the script download_mes_aides_tests.py.