528 lines
20 KiB
Plaintext
528 lines
20 KiB
Plaintext
=======================
|
|
Advanced testing topics
|
|
=======================
|
|
|
|
The request factory
|
|
===================
|
|
|
|
.. module:: django.test.client
|
|
|
|
.. class:: RequestFactory
|
|
|
|
The :class:`~django.test.client.RequestFactory` shares the same API as
|
|
the test client. However, instead of behaving like a browser, the
|
|
RequestFactory provides a way to generate a request instance that can
|
|
be used as the first argument to any view. This means you can test a
|
|
view function the same way as you would test any other function -- as
|
|
a black box, with exactly known inputs, testing for specific outputs.
|
|
|
|
The API for the :class:`~django.test.client.RequestFactory` is a slightly
|
|
restricted subset of the test client API:
|
|
|
|
* It only has access to the HTTP methods :meth:`~Client.get()`,
|
|
:meth:`~Client.post()`, :meth:`~Client.put()`,
|
|
:meth:`~Client.delete()`, :meth:`~Client.head()` and
|
|
:meth:`~Client.options()`.
|
|
|
|
* These methods accept all the same arguments *except* for
|
|
``follows``. Since this is just a factory for producing
|
|
requests, it's up to you to handle the response.
|
|
|
|
* It does not support middleware. Session and authentication
|
|
attributes must be supplied by the test itself if required
|
|
for the view to function properly.
|
|
|
|
Example
|
|
-------
|
|
|
|
The following is a simple unit test using the request factory::
|
|
|
|
from django.contrib.auth.models import User
|
|
from django.test import TestCase
|
|
from django.test.client import RequestFactory
|
|
|
|
class SimpleTest(TestCase):
|
|
def setUp(self):
|
|
# Every test needs access to the request factory.
|
|
self.factory = RequestFactory()
|
|
self.user = User.objects.create_user(
|
|
first_name='jacob', email='jacob@…', password='top_secret')
|
|
|
|
def test_details(self):
|
|
# Create an instance of a GET request.
|
|
request = self.factory.get('/customer/details')
|
|
|
|
# Recall that middleware are not suported. You can simulate a
|
|
# logged-in user by setting request.user manually.
|
|
request.user = self.user
|
|
|
|
# Test my_view() as if it were deployed at /customer/details
|
|
response = my_view(request)
|
|
self.assertEqual(response.status_code, 200)
|
|
|
|
.. _topics-testing-advanced-multidb:
|
|
|
|
Tests and multiple databases
|
|
============================
|
|
|
|
.. _topics-testing-masterslave:
|
|
|
|
Testing master/slave configurations
|
|
-----------------------------------
|
|
|
|
If you're testing a multiple database configuration with master/slave
|
|
replication, this strategy of creating test databases poses a problem.
|
|
When the test databases are created, there won't be any replication,
|
|
and as a result, data created on the master won't be seen on the
|
|
slave.
|
|
|
|
To compensate for this, Django allows you to define that a database is
|
|
a *test mirror*. Consider the following (simplified) example database
|
|
configuration::
|
|
|
|
DATABASES = {
|
|
'default': {
|
|
'ENGINE': 'django.db.backends.mysql',
|
|
'NAME': 'myproject',
|
|
'HOST': 'dbmaster',
|
|
# ... plus some other settings
|
|
},
|
|
'slave': {
|
|
'ENGINE': 'django.db.backends.mysql',
|
|
'NAME': 'myproject',
|
|
'HOST': 'dbslave',
|
|
'TEST_MIRROR': 'default'
|
|
# ... plus some other settings
|
|
}
|
|
}
|
|
|
|
In this setup, we have two database servers: ``dbmaster``, described
|
|
by the database alias ``default``, and ``dbslave`` described by the
|
|
alias ``slave``. As you might expect, ``dbslave`` has been configured
|
|
by the database administrator as a read slave of ``dbmaster``, so in
|
|
normal activity, any write to ``default`` will appear on ``slave``.
|
|
|
|
If Django created two independent test databases, this would break any
|
|
tests that expected replication to occur. However, the ``slave``
|
|
database has been configured as a test mirror (using the
|
|
:setting:`TEST_MIRROR` setting), indicating that under testing,
|
|
``slave`` should be treated as a mirror of ``default``.
|
|
|
|
When the test environment is configured, a test version of ``slave``
|
|
will *not* be created. Instead the connection to ``slave``
|
|
will be redirected to point at ``default``. As a result, writes to
|
|
``default`` will appear on ``slave`` -- but because they are actually
|
|
the same database, not because there is data replication between the
|
|
two databases.
|
|
|
|
.. _topics-testing-creation-dependencies:
|
|
|
|
Controlling creation order for test databases
|
|
---------------------------------------------
|
|
|
|
By default, Django will assume all databases depend on the ``default``
|
|
database and therefore always create the ``default`` database first.
|
|
However, no guarantees are made on the creation order of any other
|
|
databases in your test setup.
|
|
|
|
If your database configuration requires a specific creation order, you
|
|
can specify the dependencies that exist using the
|
|
:setting:`TEST_DEPENDENCIES` setting. Consider the following
|
|
(simplified) example database configuration::
|
|
|
|
DATABASES = {
|
|
'default': {
|
|
# ... db settings
|
|
'TEST_DEPENDENCIES': ['diamonds']
|
|
},
|
|
'diamonds': {
|
|
# ... db settings
|
|
'TEST_DEPENDENCIES': []
|
|
},
|
|
'clubs': {
|
|
# ... db settings
|
|
'TEST_DEPENDENCIES': ['diamonds']
|
|
},
|
|
'spades': {
|
|
# ... db settings
|
|
'TEST_DEPENDENCIES': ['diamonds','hearts']
|
|
},
|
|
'hearts': {
|
|
# ... db settings
|
|
'TEST_DEPENDENCIES': ['diamonds','clubs']
|
|
}
|
|
}
|
|
|
|
Under this configuration, the ``diamonds`` database will be created first,
|
|
as it is the only database alias without dependencies. The ``default`` and
|
|
``clubs`` alias will be created next (although the order of creation of this
|
|
pair is not guaranteed); then ``hearts``; and finally ``spades``.
|
|
|
|
If there are any circular dependencies in the
|
|
:setting:`TEST_DEPENDENCIES` definition, an ``ImproperlyConfigured``
|
|
exception will be raised.
|
|
|
|
Advanced features of ``TransactionTestCase``
|
|
============================================
|
|
|
|
.. currentmodule:: django.test
|
|
|
|
.. attribute:: TransactionTestCase.available_apps
|
|
|
|
.. versionadded:: 1.6
|
|
|
|
.. warning::
|
|
|
|
This attribute is a private API. It may be changed or removed without
|
|
a deprecation period in the future, for instance to accommodate changes
|
|
in application loading.
|
|
|
|
It's used to optimize Django's own test suite, which contains hundreds
|
|
of models but no relations between models in different applications.
|
|
|
|
By default, ``available_apps`` is set to ``None``. After each test, Django
|
|
calls :djadmin:`flush` to reset the database state. This empties all tables
|
|
and emits the :data:`~django.db.models.signals.post_migrate` signal, which
|
|
re-creates one content type and three permissions for each model. This
|
|
operation gets expensive proportionally to the number of models.
|
|
|
|
Setting ``available_apps`` to a list of applications instructs Django to
|
|
behave as if only the models from these applications were available. The
|
|
behavior of ``TransactionTestCase`` changes as follows:
|
|
|
|
- :data:`~django.db.models.signals.post_migrate` is fired before each
|
|
test to create the content types and permissions for each model in
|
|
available apps, in case they're missing.
|
|
- After each test, Django empties only tables corresponding to models in
|
|
available apps. However, at the database level, truncation may cascade to
|
|
related models in unavailable apps. Furthermore
|
|
:data:`~django.db.models.signals.post_migrate` isn't fired; it will be
|
|
fired by the next ``TransactionTestCase``, after the correct set of
|
|
applications is selected.
|
|
|
|
Since the database isn't fully flushed, if a test creates instances of
|
|
models not included in ``available_apps``, they will leak and they may
|
|
cause unrelated tests to fail. Be careful with tests that use sessions;
|
|
the default session engine stores them in the database.
|
|
|
|
Since :data:`~django.db.models.signals.post_migrate` isn't emitted after
|
|
flushing the database, its state after a ``TransactionTestCase`` isn't the
|
|
same as after a ``TestCase``: it's missing the rows created by listeners
|
|
to :data:`~django.db.models.signals.post_migrate`. Considering the
|
|
:ref:`order in which tests are executed <order-of-tests>`, this isn't an
|
|
issue, provided either all ``TransactionTestCase`` in a given test suite
|
|
declare ``available_apps``, or none of them.
|
|
|
|
``available_apps`` is mandatory in Django's own test suite.
|
|
|
|
.. attribute:: TransactionTestCase.reset_sequences
|
|
|
|
Setting ``reset_sequences = True`` on a ``TransactionTestCase`` will make
|
|
sure sequences are always reset before the test run::
|
|
|
|
class TestsThatDependsOnPrimaryKeySequences(TransactionTestCase):
|
|
reset_sequences = True
|
|
|
|
def test_animal_pk(self):
|
|
lion = Animal.objects.create(name="lion", sound="roar")
|
|
# lion.pk is guaranteed to always be 1
|
|
self.assertEqual(lion.pk, 1)
|
|
|
|
Unless you are explicitly testing primary keys sequence numbers, it is
|
|
recommended that you do not hard code primary key values in tests.
|
|
|
|
Using ``reset_sequences = True`` will slow down the test, since the primary
|
|
key reset is an relatively expensive database operation.
|
|
|
|
Running tests outside the test runner
|
|
=====================================
|
|
|
|
If you want to run tests outside of ``./manage.py test`` -- for example,
|
|
from a shell prompt -- you will need to set up the test
|
|
environment first. Django provides a convenience method to do this::
|
|
|
|
>>> from django.test.utils import setup_test_environment
|
|
>>> setup_test_environment()
|
|
|
|
:func:`~django.test.utils.setup_test_environment` puts several Django features
|
|
into modes that allow for repeatable testing, but does not create the test
|
|
databases; :func:`django.test.runner.DiscoverRunner.setup_databases`
|
|
takes care of that.
|
|
|
|
The call to :func:`~django.test.utils.setup_test_environment` is made
|
|
automatically as part of the setup of ``./manage.py test``. You only
|
|
need to manually invoke this method if you're not using running your
|
|
tests via Django's test runner.
|
|
|
|
.. _other-testing-frameworks:
|
|
|
|
Using different testing frameworks
|
|
==================================
|
|
|
|
Clearly, :mod:`unittest` is not the only Python testing framework. While Django
|
|
doesn't provide explicit support for alternative frameworks, it does provide a
|
|
way to invoke tests constructed for an alternative framework as if they were
|
|
normal Django tests.
|
|
|
|
When you run ``./manage.py test``, Django looks at the :setting:`TEST_RUNNER`
|
|
setting to determine what to do. By default, :setting:`TEST_RUNNER` points to
|
|
``'django.test.runner.DiscoverRunner'``. This class defines the default Django
|
|
testing behavior. This behavior involves:
|
|
|
|
#. Performing global pre-test setup.
|
|
|
|
#. Looking for tests in any file below the current directory whose name matches
|
|
the pattern ``test*.py``.
|
|
|
|
#. Creating the test databases.
|
|
|
|
#. Running ``migrate`` to install models and initial data into the test
|
|
databases.
|
|
|
|
#. Running the tests that were found.
|
|
|
|
#. Destroying the test databases.
|
|
|
|
#. Performing global post-test teardown.
|
|
|
|
If you define your own test runner class and point :setting:`TEST_RUNNER` at
|
|
that class, Django will execute your test runner whenever you run
|
|
``./manage.py test``. In this way, it is possible to use any test framework
|
|
that can be executed from Python code, or to modify the Django test execution
|
|
process to satisfy whatever testing requirements you may have.
|
|
|
|
.. _topics-testing-test_runner:
|
|
|
|
Defining a test runner
|
|
----------------------
|
|
|
|
.. currentmodule:: django.test.runner
|
|
|
|
A test runner is a class defining a ``run_tests()`` method. Django ships
|
|
with a ``DiscoverRunner`` class that defines the default Django
|
|
testing behavior. This class defines the ``run_tests()`` entry point,
|
|
plus a selection of other methods that are used to by ``run_tests()`` to
|
|
set up, execute and tear down the test suite.
|
|
|
|
.. class:: DiscoverRunner(pattern='test*.py', top_level=None, verbosity=1, interactive=True, failfast=True, **kwargs)
|
|
|
|
``DiscoverRunner`` will search for tests in any file matching ``pattern``.
|
|
|
|
``top_level`` can be used to specify the directory containing your
|
|
top-level Python modules. Usually Django can figure this out automatically,
|
|
so it's not necessary to specify this option. If specified, it should
|
|
generally be the directory containing your ``manage.py`` file.
|
|
|
|
``verbosity`` determines the amount of notification and debug information
|
|
that will be printed to the console; ``0`` is no output, ``1`` is normal
|
|
output, and ``2`` is verbose output.
|
|
|
|
If ``interactive`` is ``True``, the test suite has permission to ask the
|
|
user for instructions when the test suite is executed. An example of this
|
|
behavior would be asking for permission to delete an existing test
|
|
database. If ``interactive`` is ``False``, the test suite must be able to
|
|
run without any manual intervention.
|
|
|
|
If ``failfast`` is ``True``, the test suite will stop running after the
|
|
first test failure is detected.
|
|
|
|
Django may, from time to time, extend the capabilities of the test runner
|
|
by adding new arguments. The ``**kwargs`` declaration allows for this
|
|
expansion. If you subclass ``DiscoverRunner`` or write your own test
|
|
runner, ensure it accepts ``**kwargs``.
|
|
|
|
Your test runner may also define additional command-line options.
|
|
If you add an ``option_list`` attribute to a subclassed test runner,
|
|
those options will be added to the list of command-line options that
|
|
the :djadmin:`test` command can use.
|
|
|
|
Attributes
|
|
~~~~~~~~~~
|
|
|
|
.. attribute:: DiscoverRunner.option_list
|
|
|
|
This is the tuple of ``optparse`` options which will be fed into the
|
|
management command's ``OptionParser`` for parsing arguments. See the
|
|
documentation for Python's ``optparse`` module for more details.
|
|
|
|
Methods
|
|
~~~~~~~
|
|
|
|
.. method:: DiscoverRunner.run_tests(test_labels, extra_tests=None, **kwargs)
|
|
|
|
Run the test suite.
|
|
|
|
``test_labels`` is a list of strings describing the tests to be run. A test
|
|
label can take one of four forms:
|
|
|
|
* ``path.to.test_module.TestCase.test_method`` -- Run a single test method
|
|
in a test case.
|
|
* ``path.to.test_module.TestCase`` -- Run all the test methods in a test
|
|
case.
|
|
* ``path.to.module`` -- Search for and run all tests in the named Python
|
|
package or module.
|
|
* ``path/to/directory`` -- Search for and run all tests below the named
|
|
directory.
|
|
|
|
If ``test_labels`` has a value of ``None``, the test runner will search for
|
|
tests in all files below the current directory whose names match its
|
|
``pattern`` (see above).
|
|
|
|
``extra_tests`` is a list of extra ``TestCase`` instances to add to the
|
|
suite that is executed by the test runner. These extra tests are run
|
|
in addition to those discovered in the modules listed in ``test_labels``.
|
|
|
|
This method should return the number of tests that failed.
|
|
|
|
.. method:: DiscoverRunner.setup_test_environment(**kwargs)
|
|
|
|
Sets up the test environment by calling
|
|
:func:`~django.test.utils.setup_test_environment` and setting
|
|
:setting:`DEBUG` to ``False``.
|
|
|
|
.. method:: DiscoverRunner.build_suite(test_labels, extra_tests=None, **kwargs)
|
|
|
|
Constructs a test suite that matches the test labels provided.
|
|
|
|
``test_labels`` is a list of strings describing the tests to be run. A test
|
|
label can take one of three forms:
|
|
|
|
* ``app.TestCase.test_method`` -- Run a single test method in a test
|
|
case.
|
|
* ``app.TestCase`` -- Run all the test methods in a test case.
|
|
* ``app`` -- Search for and run all tests in the named application.
|
|
|
|
If ``test_labels`` has a value of ``None``, the test runner should run
|
|
search for tests in all the applications in :setting:`INSTALLED_APPS`.
|
|
|
|
``extra_tests`` is a list of extra ``TestCase`` instances to add to the
|
|
suite that is executed by the test runner. These extra tests are run
|
|
in addition to those discovered in the modules listed in ``test_labels``.
|
|
|
|
Returns a ``TestSuite`` instance ready to be run.
|
|
|
|
.. method:: DiscoverRunner.setup_databases(**kwargs)
|
|
|
|
Creates the test databases.
|
|
|
|
Returns a data structure that provides enough detail to undo the changes
|
|
that have been made. This data will be provided to the ``teardown_databases()``
|
|
function at the conclusion of testing.
|
|
|
|
.. method:: DiscoverRunner.run_suite(suite, **kwargs)
|
|
|
|
Runs the test suite.
|
|
|
|
Returns the result produced by the running the test suite.
|
|
|
|
.. method:: DiscoverRunner.teardown_databases(old_config, **kwargs)
|
|
|
|
Destroys the test databases, restoring pre-test conditions.
|
|
|
|
``old_config`` is a data structure defining the changes in the
|
|
database configuration that need to be reversed. It is the return
|
|
value of the ``setup_databases()`` method.
|
|
|
|
.. method:: DiscoverRunner.teardown_test_environment(**kwargs)
|
|
|
|
Restores the pre-test environment.
|
|
|
|
.. method:: DiscoverRunner.suite_result(suite, result, **kwargs)
|
|
|
|
Computes and returns a return code based on a test suite, and the result
|
|
from that test suite.
|
|
|
|
|
|
Testing utilities
|
|
-----------------
|
|
|
|
django.test.utils
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
.. module:: django.test.utils
|
|
:synopsis: Helpers to write custom test runners.
|
|
|
|
To assist in the creation of your own test runner, Django provides a number of
|
|
utility methods in the ``django.test.utils`` module.
|
|
|
|
.. function:: setup_test_environment()
|
|
|
|
Performs any global pre-test setup, such as the installing the
|
|
instrumentation of the template rendering system and setting up
|
|
the dummy email outbox.
|
|
|
|
.. function:: teardown_test_environment()
|
|
|
|
Performs any global post-test teardown, such as removing the black
|
|
magic hooks into the template system and restoring normal email
|
|
services.
|
|
|
|
django.db.connection.creation
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. currentmodule:: django.db.connection.creation
|
|
|
|
The creation module of the database backend also provides some utilities that
|
|
can be useful during testing.
|
|
|
|
.. function:: create_test_db([verbosity=1, autoclobber=False])
|
|
|
|
Creates a new test database and runs ``migrate`` against it.
|
|
|
|
``verbosity`` has the same behavior as in ``run_tests()``.
|
|
|
|
``autoclobber`` describes the behavior that will occur if a
|
|
database with the same name as the test database is discovered:
|
|
|
|
* If ``autoclobber`` is ``False``, the user will be asked to
|
|
approve destroying the existing database. ``sys.exit`` is
|
|
called if the user does not approve.
|
|
|
|
* If autoclobber is ``True``, the database will be destroyed
|
|
without consulting the user.
|
|
|
|
Returns the name of the test database that it created.
|
|
|
|
``create_test_db()`` has the side effect of modifying the value of
|
|
:setting:`NAME` in :setting:`DATABASES` to match the name of the test
|
|
database.
|
|
|
|
.. function:: destroy_test_db(old_database_name, [verbosity=1])
|
|
|
|
Destroys the database whose name is the value of :setting:`NAME` in
|
|
:setting:`DATABASES`, and sets :setting:`NAME` to the value of
|
|
``old_database_name``.
|
|
|
|
The ``verbosity`` argument has the same behavior as for
|
|
:class:`~django.test.runner.DiscoverRunner`.
|
|
|
|
.. _topics-testing-code-coverage:
|
|
|
|
Integration with coverage.py
|
|
============================
|
|
|
|
Code coverage describes how much source code has been tested. It shows which
|
|
parts of your code are being exercised by tests and which are not. It's an
|
|
important part of testing applications, so it's strongly recommended to check
|
|
the coverage of your tests.
|
|
|
|
Django can be easily integrated with `coverage.py`_, a tool for measuring code
|
|
coverage of Python programs. First, `install coverage.py`_. Next, run the
|
|
following from your project folder containing ``manage.py``::
|
|
|
|
coverage run --source='.' manage.py test myapp
|
|
|
|
This runs your tests and collects coverage data of the executed files in your
|
|
project. You can see a report of this data by typing following command::
|
|
|
|
coverage report
|
|
|
|
Note that some Django code was executed while running tests, but it is not
|
|
listed here because of the ``source`` flag passed to the previous command.
|
|
|
|
For more options like annotated HTML listings detailing missed lines, see the
|
|
`coverage.py`_ docs.
|
|
|
|
.. _coverage.py: http://nedbatchelder.com/code/coverage/
|
|
.. _install coverage.py: http://pypi.python.org/pypi/coverage
|