Refs #2333 - Added more documentation for testing framework, and clarified some code as a result of trying to describe it.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@3689 bcc190cf-cafb-0310-a4f2-bffc1f526a37

django/test/simple.py

@@ -61,7 +61,8 @@ def run_tests(module_list, verbosity=1, extra_tests=[]):
     for test in extra_tests:
         suite.addTest(test)
 
-    old_name = create_test_db(verbosity)
+    old_name = settings.DATABASE_NAME
+    create_test_db(verbosity)
     management.syncdb(verbosity, interactive=False)
     unittest.TextTestRunner(verbosity=verbosity).run(suite)
     destroy_test_db(old_name, verbosity)

django/test/utils.py

@@ -50,14 +50,11 @@ def create_test_db(verbosity=1, autoclobber=False):
                 sys.exit(1)
                
     connection.close()
-    old_database_name = settings.DATABASE_NAME
     settings.DATABASE_NAME = TEST_DATABASE_NAME
 
     # Get a cursor (even though we don't need one yet). This has
     # the side effect of initializing the test database.
     cursor = connection.cursor()
-            
-    return old_database_name
 
 def destroy_test_db(old_database_name, verbosity=1):
     # Unless we're using SQLite, remove the test database to clean up after

docs/django-admin.txt

@@ -345,6 +345,17 @@ setting the Python path for you.
 Displays a help message that includes a terse list of all available actions and
 options.
 
+--noinput
+---------
+
+Inform django-admin that the user should NOT be prompted for any input. Useful if
+the django-admin script will be executed as an unattended, automated script.
+
+--noreload
+----------
+
+Disable the use of the auto-reloader when running the development server.
+
 --version
 ---------
 
@@ -355,6 +366,17 @@ Example output::
     0.9.1
     0.9.1 (SVN)
 
+--verbosity
+-----------
+
+Example usage::
+
+    django-admin.py syncdb --verbosity=2
+
+Verbosity determines the amount of notification and debug information that 
+will be printed to the console. '0' is no output, '1' is normal output,
+and `2` is verbose output.
+
 Extra niceties
 ==============
 

docs/settings.txt

@@ -754,6 +754,18 @@ misspelled) variables. See `How invalid variables are handled`_.
 
 .. _How invalid variables are handled: http://www.djangoproject.com/documentation/templates_python/#how-invalid-variables-are-handled
 
+TEST_RUNNER
+-----------
+
+**New in Django development version**
+
+Default: ``'django.test.simple.run_tests'``
+
+The name of the method to use for starting the test suite. See 
+`Testing Django Applications`_.
+
+.. _Testing Django Applications: ../testing/
+
 TIME_FORMAT
 -----------
 

docs/testing.txt

@@ -4,13 +4,21 @@ Testing Django applications
 
 **New in Django development version**.
 
-.. XXX insert quick introduction to testing (and why you'd want to do it)
+Automated testing is an extremely useful weapon in the bug-killing arsenal
+of the modern developer. When initially writing code, a test suite can be
+used to validate that code behaves as expected. When refactoring or
+modifying code, tests serve as a guide to ensure that behavior hasn't
+changed as a result of the refactor.
+
+Testing an web application is a complex task, as there are many
+components of a web application that must be validated and tested. To
+help you test your application, Django provides a test execution
+framework, and range of utilities that can be used to stimulate and
+inspect various facets of a web application.
 
-.. note::
-    
     This testing framework is currently under development, and may change
-    slightly before the next official Django release.  
-    
+    slightly before the next official Django release.
+
     (That's *no* excuse not to write tests, though!)
 
 Writing tests
@@ -23,17 +31,20 @@ Writing doctests
 
 Doctests use Python's standard doctest_ module, which searches for tests in
 your docstrings. Django's test runner looks for doctests in your ``models.py``
-file, and executes any that it finds.
+file, and executes any that it finds. Django will also search for a file
+called ``tests.py`` in the application directory (i.e., the directory that
+holds ``models.py``). If a ``tests.py`` is found, it will also be searched
+for doctests.
 
 .. admonition:: What's a **docstring**?
 
     A good explanation of docstrings (and some guidlines for using them
     effectively) can be found in :PEP:`257`:
-        
-        A docstring is a string literal that occurs as the first statement in 
+
+        A docstring is a string literal that occurs as the first statement in
         a module, function, class, or method definition.  Such a docstring
         becomes the ``__doc__`` special attribute of that object.
-        
+
     Since tests often make great documentation, doctest lets you put your
     tests directly in your docstrings.
 
@@ -44,25 +55,25 @@ model-level doctests in the docstring for each model.
 For example::
 
     from django.db import model
-    
+
     class Animal(models.Model):
         """
         An animal that knows how to make noise
-        
+
         # Create some animals
         >>> lion = Animal.objects.create(name="lion", sound="roar")
         >>> cat = Animal.objects.create(name="cat", sound="meow")
-        
+
         # Make 'em speak
         >>> lion.speak()
         'The lion says "roar"'
         >>> cat.speak()
         'The cat says "meow"'
         """
-        
+
         name = models.CharField(maxlength=20)
         sound = models.CharField(maxlength=20)
-        
+
         def speak(self):
             return 'The %s says "%s"' % (self.name, self.sound)
 
@@ -80,24 +91,24 @@ Writing unittests
 -----------------
 
 Like doctests, Django's unit tests use a standard library module: unittest_.
-Django's test runner looks for unit test cases in a ``tests.py`` file in your
-app (i.e. in the same directory as your ``models.py`` file).
+As with doctests, Django's test runner looks for any unit test cases defined
+in ``models.py``, or in a ``tests.py`` file in your application directory.
 
 An equivalent unittest test case for the above example would look like::
 
     import unittest
     from myapp.models import Animal
-    
+
     class AnimalTestCase(unittest.TestCase):
-        
+
         def setUp(self):
             self.lion = Animal.objects.create(name="lion", sound="roar")
             self.cat = Animal.objects.create(name="cat", sound="meow")
-        
+
         def testSpeaking(self):
             self.assertEquals(self.lion.speak(), 'The lion says "roar"')
             self.assertEquals(self.cat.speak(), 'The cat says "meow"')
-            
+
 When you `run your tests`_, the test utility will find all the test cases
 (that is, subclasses of ``unittest.TestCase``) in ``tests.py``, automatically
 build a test suite out of those test cases, and run that suite.
@@ -119,7 +130,7 @@ system has different benefits, the best approach is probably to use both
 together, picking the test system to match the type of tests you need to
 write.
 
-For developers new to testing, however, this choice can seem 
+For developers new to testing, however, this choice can seem
 confusing, so here are a few key differences to help you decide weather
 doctests or unit tests are right for you.
 
@@ -136,11 +147,11 @@ get you started faster.
 The ``unittest`` framework will probably feel very familiar to developers
 coming from Java.  Since ``unittest`` is inspired by Java's JUnit, if
 you've used testing frameworks in other languages that similarly were
-inspired by JUnit, ``unittest`` should also feel pretty familiar.  
+inspired by JUnit, ``unittest`` should also feel pretty familiar.
 
 Since ``unittest`` is organized around classes and methods, if you need
 to write a bunch of tests that all share similar code, you can easily use
-subclass to abstract common tasks; this makes test code shorter and cleaner. 
+subclass to abstract common tasks; this makes test code shorter and cleaner.
 There's also support for explicit setup and/or cleanup routines, which give
 you a high level of control over the environment your test cases run in.
 
@@ -148,6 +159,20 @@ Again, remember that you can use both systems side-by-side (even in the same
 app). In the end, most projects will eventually end up using both; each shines
 in different circumstances.
 
+Testing utilities
+=================
+
+Test Client
+-----------
+
+A dummy browser; instruments the template generation process...
+
+Fixtures
+--------
+
+Feature still to come...
+
+
 Running tests
 =============
 
@@ -155,9 +180,22 @@ Run your tests using your project's ``manage.py`` utility::
 
     $ ./manage.py test
 
-You'll see a bunch of text flow by as the test database is created, models are
-initialized, and your tests are run. If everything goes well, at the end
-you'll see::
+If you only want to run tests for a particular application, add the
+application name to the command line. For example, if your
+``INSTALLED_APPS`` contains ``myproject.polls`` and ``myproject.animals``,
+but you only want to run the animals unit tests, run::
+
+    $ ./manage.py test animals
+
+When you run your tests, you'll see a bunch of text flow by as the test
+database is created and models are initialized. This test database is
+created from scratch every time you run your tests. The test database
+gets its name by prepending ``test_`` to the database name specified by
+``settings.DATABASE_NAME``; all other database settings will the same as
+they would be for the project normally.
+
+Once the test database has been established, Django will run your tests.
+If everything goes well, at the end you'll see::
 
     ----------------------------------------------------------------------
     Ran 22 tests in 0.221s
@@ -189,4 +227,74 @@ failed::
     Ran 2 tests in 0.048s
 
     FAILED (failures=1)
-    
+
+When the tests have all been executed, the test database is destroyed.
+
+Using a different testing framework
+===================================
+
+Doctest and Unittest are not the only Python testing frameworks. While
+Django doesn't provide explicit support these alternative frameworks,
+it does provide a mechanism to allow you to invoke tests constructed for
+an alternative framework as if they were normal Django tests.
+
+When you run ``./manage.py test``, Django looks at the ``TEST_RUNNER``
+setting to determine what to do. By default, ``TEST_RUNNER`` points to ``django.test.simple.run_tests``. This method defines the default Django
+testing behaviour. This behaviour involves:
+
+#. Creating the test database
+#. Running ``syncdb`` to install models and initial data into the test database
+#. Looking for Unit Tests and Doctests in ``models.py`` and ``tests.py`` file for each installed application
+#. Running the Unit Tests and Doctests that are found
+#. Destroying the test database.
+
+If you define your own test runner method and point ``TEST_RUNNER``
+at that method, Django will execute your test runner whenever you run
+``./manage.py test``. In this way, it is possible to use any test
+framework that can be executed from Python code.
+
+Defining a test runner
+----------------------
+By convention, a test runner should be called ``run_tests``; however, you
+can call it anything you want. The only requirement is that it accept two
+arguments:
+
+``run_tests(module_list, verbosity=1)``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The module list is the list of Python modules that contain the models to be
+tested. This is the same format returned by ``django.db.models.get_apps()``
+
+Verbosity determines the amount of debug information that will be
+provided to the console; '0' is no output, '1' is normal output,
+and `2` is verbose output.
+
+Testing utilities
+-----------------
+
+To assist in the creation of your own test runner, Django provides
+a number of utility methods in the ``django.test.utils`` module.
+
+``create_test_db(verbosity=1, autoclobber=False)``:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Creates a new test database, and run ``syncdb`` against it.
+
+``verbosity`` has the same behaviour as in the test runner.
+
+``Autoclobber`` describes the behavior that will occur if a database with
+the same name as the test database is discovered. If ``autoclobber`` is False,
+the user will be asked to approve destroying the existing database. ``sys.exit``
+is called if the user does not approve. If autoclobber is ``True``, the database
+will be destroyed without consulting the user.
+
+``create_test_db()`` has the side effect of modifying
+``settings.DATABASE_NAME`` to match the name of the test database.
+
+``destroy_test_db(old_database_name, verbosity=1)``:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Destroys the database with the name ``settings.DATABASE_NAME`` matching,
+and restores the value of ``settings.DATABASE_NAME`` to the provided name.
+
+``verbosity`` has the same behaviour as in the test runner.