420 lines
15 KiB
Plaintext
420 lines
15 KiB
Plaintext
==========
|
|
Unit tests
|
|
==========
|
|
|
|
.. highlight:: console
|
|
|
|
Django comes with a test suite of its own, in the ``tests`` directory of the
|
|
code base. It's our policy to make sure all tests pass at all times.
|
|
|
|
We appreciate any and all contributions to the test suite!
|
|
|
|
The Django tests all use the testing infrastructure that ships with Django for
|
|
testing applications. See :doc:`/topics/testing/overview` for an explanation of
|
|
how to write new tests.
|
|
|
|
.. _running-unit-tests:
|
|
|
|
Running the unit tests
|
|
======================
|
|
|
|
Quickstart
|
|
----------
|
|
|
|
If you are on Python 2, you'll first need to install a backport of the
|
|
``unittest.mock`` module that's available in Python 3. See
|
|
:ref:`running-unit-tests-dependencies` for details on installing `mock`_ and
|
|
the other optional test dependencies.
|
|
|
|
Running the tests requires a Django settings module that defines the
|
|
databases to use. To make it easy to get started, Django provides and uses a
|
|
sample settings module that uses the SQLite database. To run the tests::
|
|
|
|
$ git clone https://github.com/django/django.git django-repo
|
|
$ cd django-repo/tests
|
|
$ PYTHONPATH=..:$PYTHONPATH ./runtests.py
|
|
|
|
.. admonition:: Windows users
|
|
|
|
We recommend something like `Git Bash <https://msysgit.github.io/>`_ to run
|
|
the tests using the above approach.
|
|
|
|
You can avoid typing the ``PYTHONPATH`` bit each time by adding your Django
|
|
checkout to your ``PYTHONPATH`` or by installing the source checkout using pip.
|
|
See :ref:`installing-development-version`.
|
|
|
|
Having problems? See :ref:`troubleshooting-unit-tests` for some common issues.
|
|
|
|
.. _running-unit-tests-settings:
|
|
|
|
Using another ``settings`` module
|
|
---------------------------------
|
|
|
|
The included settings module allows you to run the test suite using
|
|
SQLite. If you want to test behavior using a different database (and
|
|
if you're proposing patches for Django, it's a good idea to test
|
|
across databases), you may need to define your own settings file.
|
|
|
|
To run the tests with different settings, ensure that the module is on your
|
|
``PYTHONPATH`` and pass the module with ``--settings``.
|
|
|
|
The :setting:`DATABASES` setting in any test settings module needs to define
|
|
two databases:
|
|
|
|
* A ``default`` database. This database should use the backend that
|
|
you want to use for primary testing.
|
|
|
|
* A database with the alias ``other``. The ``other`` database is used to
|
|
establish that queries can be directed to different databases. As a result,
|
|
this database can use any backend you want. It doesn't need to use the same
|
|
backend as the ``default`` database (although it can use the same backend if
|
|
you want to). It cannot be the same database as the ``default``.
|
|
|
|
If you're using a backend that isn't SQLite, you will need to provide other
|
|
details for each database:
|
|
|
|
* The :setting:`USER` option needs to specify an existing user account
|
|
for the database. That user needs permission to execute ``CREATE DATABASE``
|
|
so that the test database can be created.
|
|
|
|
* The :setting:`PASSWORD` option needs to provide the password for
|
|
the :setting:`USER` that has been specified.
|
|
|
|
Test databases get their names by prepending ``test_`` to the value of the
|
|
:setting:`NAME` settings for the databases defined in :setting:`DATABASES`.
|
|
These test databases are deleted when the tests are finished.
|
|
|
|
You will also need to ensure that your database uses UTF-8 as the default
|
|
character set. If your database server doesn't use UTF-8 as a default charset,
|
|
you will need to include a value for :setting:`CHARSET <TEST_CHARSET>` in the
|
|
test settings dictionary for the applicable database.
|
|
|
|
.. _runtests-specifying-labels:
|
|
|
|
Running only some of the tests
|
|
------------------------------
|
|
|
|
Django's entire test suite takes a while to run, and running every single test
|
|
could be redundant if, say, you just added a test to Django that you want to
|
|
run quickly without running everything else. You can run a subset of the unit
|
|
tests by appending the names of the test modules to ``runtests.py`` on the
|
|
command line.
|
|
|
|
For example, if you'd like to run tests only for generic relations and
|
|
internationalization, type::
|
|
|
|
$ ./runtests.py --settings=path.to.settings generic_relations i18n
|
|
|
|
How do you find out the names of individual tests? Look in ``tests/`` — each
|
|
directory name there is the name of a test.
|
|
|
|
If you just want to run a particular class of tests, you can specify a list of
|
|
paths to individual test classes. For example, to run the ``TranslationTests``
|
|
of the ``i18n`` module, type::
|
|
|
|
$ ./runtests.py --settings=path.to.settings i18n.tests.TranslationTests
|
|
|
|
Going beyond that, you can specify an individual test method like this::
|
|
|
|
$ ./runtests.py --settings=path.to.settings i18n.tests.TranslationTests.test_lazy_objects
|
|
|
|
Running the Selenium tests
|
|
--------------------------
|
|
|
|
Some tests require Selenium and a Web browser (Firefox, Google Chrome, or
|
|
Internet Explorer). To allow those tests to be run rather than skipped, you must
|
|
install the selenium_ package into your Python path and run the tests with the
|
|
``--selenium`` option::
|
|
|
|
$ ./runtests.py --settings=test_sqlite --selenium admin_inlines
|
|
|
|
.. _running-unit-tests-dependencies:
|
|
|
|
Running all the tests
|
|
---------------------
|
|
|
|
If you want to run the full suite of tests, you'll need to install a number of
|
|
dependencies:
|
|
|
|
* bcrypt_
|
|
* docutils_
|
|
* enum34_ (Python 2 only)
|
|
* geoip2_
|
|
* jinja2_ 2.7+
|
|
* numpy_
|
|
* Pillow_
|
|
* PyYAML_
|
|
* pytz_
|
|
* setuptools_
|
|
* memcached_, plus a :ref:`supported Python binding <memcached>`
|
|
* mock_ (for Python 2)
|
|
* gettext_ (:ref:`gettext_on_windows`)
|
|
* selenium_
|
|
* sqlparse_
|
|
|
|
You can find these dependencies in `pip requirements files`_ inside the
|
|
``tests/requirements`` directory of the Django source tree and install them
|
|
like so::
|
|
|
|
$ pip install -r tests/requirements/py3.txt # Python 2: py2.txt
|
|
|
|
You can also install the database adapter(s) of your choice using
|
|
``oracle.txt``, ``mysql.txt``, or ``postgres.txt``.
|
|
|
|
If you want to test the memcached cache backend, you'll also need to define
|
|
a :setting:`CACHES` setting that points at your memcached instance.
|
|
|
|
To run the GeoDjango tests, you will need to :doc:`setup a spatial database
|
|
and install the Geospatial libraries</ref/contrib/gis/install/index>`.
|
|
|
|
Each of these dependencies is optional. If you're missing any of them, the
|
|
associated tests will be skipped.
|
|
|
|
.. _bcrypt: https://pypi.python.org/pypi/bcrypt
|
|
.. _docutils: https://pypi.python.org/pypi/docutils
|
|
.. _enum34: https://pypi.python.org/pypi/enum34
|
|
.. _geoip2: https://pypi.python.org/pypi/geoip2
|
|
.. _jinja2: https://pypi.python.org/pypi/jinja2
|
|
.. _numpy: https://pypi.python.org/pypi/numpy
|
|
.. _Pillow: https://pypi.python.org/pypi/Pillow/
|
|
.. _PyYAML: http://pyyaml.org/wiki/PyYAML
|
|
.. _pytz: https://pypi.python.org/pypi/pytz/
|
|
.. _setuptools: https://pypi.python.org/pypi/setuptools/
|
|
.. _memcached: http://memcached.org/
|
|
.. _mock: https://pypi.python.org/pypi/mock
|
|
.. _gettext: https://www.gnu.org/software/gettext/manual/gettext.html
|
|
.. _selenium: https://pypi.python.org/pypi/selenium
|
|
.. _sqlparse: https://pypi.python.org/pypi/sqlparse
|
|
.. _pip requirements files: https://pip.pypa.io/en/latest/user_guide.html#requirements-files
|
|
|
|
Code coverage
|
|
-------------
|
|
|
|
Contributors are encouraged to run coverage on the test suite to identify areas
|
|
that need additional tests. The coverage tool installation and use is described
|
|
in :ref:`testing code coverage<topics-testing-code-coverage>`.
|
|
|
|
Coverage should be run in a single process to obtain accurate statistics. To
|
|
run coverage on the Django test suite using the standard test settings::
|
|
|
|
$ coverage run ./runtests.py --settings=test_sqlite --parallel=1
|
|
|
|
After running coverage, generate the html report by running::
|
|
|
|
$ coverage html
|
|
|
|
When running coverage for the Django tests, the included ``.coveragerc``
|
|
settings file defines ``coverage_html`` as the output directory for the report
|
|
and also excludes several directories not relevant to the results
|
|
(test code or external code included in Django).
|
|
|
|
.. _contrib-apps:
|
|
|
|
Contrib apps
|
|
============
|
|
|
|
Tests for contrib apps can be found in the ``tests/`` directory, typically
|
|
under ``<app_name>_tests``. For example, tests for ``contrib.auth`` are located
|
|
in ``tests/auth_tests``.
|
|
|
|
.. _troubleshooting-unit-tests:
|
|
|
|
Troubleshooting
|
|
===============
|
|
|
|
Many test failures with ``UnicodeEncodeError``
|
|
----------------------------------------------
|
|
|
|
If the ``locales`` package is not installed, some tests will fail with a
|
|
``UnicodeEncodeError``.
|
|
|
|
You can resolve this on Debian-based systems, for example, by running::
|
|
|
|
$ apt-get install locales
|
|
$ dpkg-reconfigure locales
|
|
|
|
Tests that only fail in combination
|
|
-----------------------------------
|
|
|
|
In case a test passes when run in isolation but fails within the whole suite,
|
|
we have some tools to help analyze the problem.
|
|
|
|
The ``--bisect`` option of ``runtests.py`` will run the failing test while
|
|
halving the test set it is run together with on each iteration, often making
|
|
it possible to identify a small number of tests that may be related to the
|
|
failure.
|
|
|
|
For example, suppose that the failing test that works on its own is
|
|
``ModelTest.test_eq``, then using::
|
|
|
|
$ ./runtests.py --bisect basic.tests.ModelTest.test_eq
|
|
|
|
will try to determine a test that interferes with the given one. First, the
|
|
test is run with the first half of the test suite. If a failure occurs, the
|
|
first half of the test suite is split in two groups and each group is then run
|
|
with the specified test. If there is no failure with the first half of the test
|
|
suite, the second half of the test suite is run with the specified test and
|
|
split appropriately as described earlier. The process repeats until the set of
|
|
failing tests is minimized.
|
|
|
|
The ``--pair`` option runs the given test alongside every other test from the
|
|
suite, letting you check if another test has side-effects that cause the
|
|
failure. So::
|
|
|
|
$ ./runtests.py --pair basic.tests.ModelTest.test_eq
|
|
|
|
will pair ``test_eq`` with every test label.
|
|
|
|
With both ``--bisect`` and ``--pair``, if you already suspect which cases
|
|
might be responsible for the failure, you may limit tests to be cross-analyzed
|
|
by :ref:`specifying further test labels <runtests-specifying-labels>` after
|
|
the first one::
|
|
|
|
$ ./runtests.py --pair basic.tests.ModelTest.test_eq queries transactions
|
|
|
|
You can also try running any set of tests in reverse using the ``--reverse``
|
|
option in order to verify that executing tests in a different order does not
|
|
cause any trouble::
|
|
|
|
$ ./runtests.py basic --reverse
|
|
|
|
Seeing the SQL queries run during a test
|
|
----------------------------------------
|
|
|
|
If you wish to examine the SQL being run in failing tests, you can turn on
|
|
:ref:`SQL logging <django-db-logger>` using the ``--debug-sql`` option. If you
|
|
combine this with ``--verbosity=2``, all SQL queries will be output::
|
|
|
|
$ ./runtests.py basic --debug-sql
|
|
|
|
Seeing the full traceback of a test failure
|
|
-------------------------------------------
|
|
|
|
By default tests are run in parallel with one process per core. When the tests
|
|
are run in parallel, however, you'll only see a truncated traceback for any
|
|
test failures. You can adjust this behavior with the ``--parallel`` option::
|
|
|
|
$ ./runtests.py basic --parallel=1
|
|
|
|
You can also use the ``DJANGO_TEST_PROCESSES`` environment variable for this
|
|
purpose.
|
|
|
|
.. versionadded:: 1.9
|
|
|
|
Support for running tests in parallel and the ``--parallel`` option were
|
|
added.
|
|
|
|
Tips for writing tests
|
|
----------------------
|
|
|
|
.. highlight:: python
|
|
|
|
Isolating model registration
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
To avoid polluting the global :attr:`~django.apps.apps` registry and prevent
|
|
unnecessary table creation, models defined in a test method should be bound to
|
|
a temporary ``Apps`` instance::
|
|
|
|
from django.apps.registry import Apps
|
|
from django.db import models
|
|
from django.test import SimpleTestCase
|
|
|
|
class TestModelDefinition(SimpleTestCase):
|
|
def test_model_definition(self):
|
|
test_apps = Apps(['app_label'])
|
|
|
|
class TestModel(models.Model):
|
|
class Meta:
|
|
apps = test_apps
|
|
...
|
|
|
|
.. function:: django.test.utils.isolate_apps(*app_labels, attr_name=None, kwarg_name=None)
|
|
|
|
.. versionadded:: 1.10
|
|
|
|
Since this pattern involves a lot of boilerplate, Django provides the
|
|
:func:`~django.test.utils.isolate_apps` decorator. It's used like this::
|
|
|
|
from django.db import models
|
|
from django.test import SimpleTestCase
|
|
from django.test.utils import isolate_apps
|
|
|
|
class TestModelDefinition(SimpleTestCase):
|
|
@isolate_apps('app_label')
|
|
def test_model_definition(self):
|
|
class TestModel(models.Model):
|
|
pass
|
|
...
|
|
|
|
.. admonition:: Setting ``app_label``
|
|
|
|
Models defined in a test method with no explicit
|
|
:attr:`~django.db.models.Options.app_label` are automatically assigned the
|
|
label of the app in which their test class is located.
|
|
|
|
In order to make sure the models defined within the context of
|
|
:func:`~django.test.utils.isolate_apps` instances are correctly
|
|
installed, you should pass the set of targeted ``app_label`` as arguments:
|
|
|
|
.. snippet::
|
|
:filename: tests/app_label/tests.py
|
|
|
|
from django.db import models
|
|
from django.test import SimpleTestCase
|
|
from django.test.utils import isolate_apps
|
|
|
|
class TestModelDefinition(SimpleTestCase):
|
|
@isolate_apps('app_label', 'other_app_label')
|
|
def test_model_definition(self):
|
|
# This model automatically receives app_label='app_label'
|
|
class TestModel(models.Model):
|
|
pass
|
|
|
|
class OtherAppModel(models.Model):
|
|
class Meta:
|
|
app_label = 'other_app_label'
|
|
...
|
|
|
|
The decorator can also be applied to classes::
|
|
|
|
from django.db import models
|
|
from django.test import SimpleTestCase
|
|
from django.test.utils import isolate_apps
|
|
|
|
@isolate_apps('app_label')
|
|
class TestModelDefinition(SimpleTestCase):
|
|
def test_model_definition(self):
|
|
class TestModel(models.Model):
|
|
pass
|
|
...
|
|
|
|
The temporary ``Apps`` instance used to isolate model registration can be
|
|
retrieved as an attribute when used as a class decorator by using the
|
|
``attr_name`` parameter::
|
|
|
|
from django.db import models
|
|
from django.test import SimpleTestCase
|
|
from django.test.utils import isolate_apps
|
|
|
|
@isolate_apps('app_label', attr_name='apps')
|
|
class TestModelDefinition(SimpleTestCase):
|
|
def test_model_definition(self):
|
|
class TestModel(models.Model):
|
|
pass
|
|
self.assertIs(self.apps.get_model('app_label', 'TestModel'), TestModel)
|
|
|
|
Or as an argument on the test method when used as a method decorator by using
|
|
the ``kwarg_name`` parameter::
|
|
|
|
from django.db import models
|
|
from django.test import SimpleTestCase
|
|
from django.test.utils import isolate_apps
|
|
|
|
class TestModelDefinition(SimpleTestCase):
|
|
@isolate_apps('app_label', kwarg_name='apps')
|
|
def test_model_definition(self, apps):
|
|
class TestModel(models.Model):
|
|
pass
|
|
self.assertIs(apps.get_model('app_label', 'TestModel'), TestModel)
|