Enabled database-level autocommit for all backends.

This is mostly a documentation change.

It has the same backwards-incompatibility consequences as those
described for PostgreSQL in a previous commit.
This commit is contained in:
Aymeric Augustin 2013-03-03 15:55:11 +01:00
parent cfc114e00e
commit 5e27debc5c
7 changed files with 238 additions and 140 deletions

View File

@ -98,6 +98,8 @@ class BaseDatabaseWrapper(object):
conn_params = self.get_connection_params()
self.connection = self.get_new_connection(conn_params)
self.init_connection_state()
if not settings.TRANSACTIONS_MANAGED:
self.set_autocommit()
connection_created.send(sender=self.__class__, connection=self)
def ensure_connection(self):

View File

@ -136,7 +136,6 @@ class DatabaseWrapper(BaseDatabaseWrapper):
self.connection.cursor().execute(
self.ops.set_time_zone_sql(), [tz])
self.connection.set_isolation_level(self.isolation_level)
self.set_autocommit(not settings.TRANSACTIONS_MANAGED)
def create_cursor(self):
cursor = self.connection.cursor()

View File

@ -69,7 +69,6 @@ even ``0``, because it doesn't make sense to maintain a connection that's
unlikely to be reused. This will help keep the number of simultaneous
connections to this database small.
The development server creates a new thread for each request it handles,
negating the effect of persistent connections.
@ -104,7 +103,8 @@ Optimizing PostgreSQL's configuration
Django needs the following parameters for its database connections:
- ``client_encoding``: ``'UTF8'``,
- ``default_transaction_isolation``: ``'read committed'``,
- ``default_transaction_isolation``: ``'read committed'`` by default,
or the value set in the connection options (see below),
- ``timezone``: ``'UTC'`` when :setting:`USE_TZ` is ``True``, value of
:setting:`TIME_ZONE` otherwise.
@ -118,30 +118,16 @@ will do some additional queries to set these parameters.
.. _ALTER ROLE: http://www.postgresql.org/docs/current/interactive/sql-alterrole.html
Transaction handling
---------------------
:doc:`By default </topics/db/transactions>`, Django runs with an open
transaction which it commits automatically when any built-in, data-altering
model function is called. The PostgreSQL backends normally operate the same as
any other Django backend in this respect.
.. _postgresql-autocommit-mode:
Autocommit mode
~~~~~~~~~~~~~~~
---------------
If your application is particularly read-heavy and doesn't make many
database writes, the overhead of a constantly open transaction can
sometimes be noticeable. For those situations, you can configure Django
to use *"autocommit"* behavior for the connection, meaning that each database
operation will normally be in its own transaction, rather than having
the transaction extend over multiple operations. In this case, you can
still manually start a transaction if you're doing something that
requires consistency across multiple database operations. The
autocommit behavior is enabled by setting the ``autocommit`` key in
the :setting:`OPTIONS` part of your database configuration in
:setting:`DATABASES`::
.. versionchanged:: 1.6
In previous versions of Django, database-level autocommit could be enabled by
setting the ``autocommit`` key in the :setting:`OPTIONS` part of your database
configuration in :setting:`DATABASES`::
DATABASES = {
# ...
@ -150,29 +136,11 @@ the :setting:`OPTIONS` part of your database configuration in
},
}
In this configuration, Django still ensures that :ref:`delete()
<topics-db-queries-delete>` and :ref:`update() <topics-db-queries-update>`
queries run inside a single transaction, so that either all the affected
objects are changed or none of them are.
.. admonition:: This is database-level autocommit
This functionality is not the same as the :ref:`autocommit
<topics-db-transactions-autocommit>` decorator. That decorator is
a Django-level implementation that commits automatically after
data changing operations. The feature enabled using the
:setting:`OPTIONS` option provides autocommit behavior at the
database adapter level. It commits after *every* operation.
If you are using this feature and performing an operation akin to delete or
updating that requires multiple operations, you are strongly recommended to
wrap you operations in manual transaction handling to ensure data consistency.
You should also audit your existing code for any instances of this behavior
before enabling this feature. It's faster, but it provides less automatic
protection for multi-call operations.
Since Django 1.6, autocommit is turned on by default. This configuration is
ignored and can be safely removed.
Isolation level
~~~~~~~~~~~~~~~
---------------
.. versionadded:: 1.6
@ -200,7 +168,7 @@ such as ``REPEATABLE READ`` or ``SERIALIZABLE``, set it in the
.. _postgresql-isolation-levels: http://www.postgresql.org/docs/devel/static/transaction-iso.html
Indexes for ``varchar`` and ``text`` columns
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--------------------------------------------
When specifying ``db_index=True`` on your model fields, Django typically
outputs a single ``CREATE INDEX`` statement. However, if the database type
@ -457,7 +425,7 @@ Savepoints
Both the Django ORM and MySQL (when using the InnoDB :ref:`storage engine
<mysql-storage-engines>`) support database :ref:`savepoints
<topics-db-transactions-savepoints>`, but this feature wasn't available in
Django until version 1.4 when such supports was added.
Django until version 1.4 when such support was added.
If you use the MyISAM storage engine please be aware of the fact that you will
receive database-generated errors if you try to use the :ref:`savepoint-related

View File

@ -814,8 +814,8 @@ generating large CSV files.
.. admonition:: Performance considerations
Django is designed for short-lived requests. Streaming responses will tie
a worker process and keep a database connection idle in transaction for
the entire duration of the response. This may result in poor performance.
a worker process for the entire duration of the response. This may result
in poor performance.
Generally speaking, you should perform expensive tasks outside of the
request-response cycle, rather than resorting to a streamed response.

View File

@ -30,6 +30,18 @@ prevention <clickjacking-prevention>` are turned on.
If the default templates don't suit your tastes, you can use :ref:`custom
project and app templates <custom-app-and-project-templates>`.
Improved transaction management
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Django's transaction management was overhauled. Database-level autocommit is
now turned on by default. This makes transaction handling more explicit and
should improve performance. The existing APIs were deprecated, and new APIs
were introduced, as described in :doc:`/topics/db/transactions`.
Please review carefully the list of :ref:`known backwards-incompatibilities
<transactions-changes-from-1.5>` to determine if you need to make changes in
your code.
Persistent database connections
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -148,6 +160,16 @@ Backwards incompatible changes in 1.6
deprecation timeline for a given feature, its removal may appear as a
backwards incompatible change.
* Database-level autocommit is enabled by default in Django 1.6. While this
doesn't change the general spirit of Django's transaction management, there
are a few known backwards-incompatibities, described in the :ref:`transaction
management docs <transactions-changes-from-1.5>`. You should review your code
to determine if you're affected.
* In previous versions, database-level autocommit was only an option for
PostgreSQL, and it was disabled by default. This option is now
:ref:`ignored <postgresql-autocommit-mode>`.
* The ``django.db.models.query.EmptyQuerySet`` can't be instantiated any more -
it is only usable as a marker class for checking if
:meth:`~django.db.models.query.QuerySet.none` has been called:

View File

@ -201,31 +201,32 @@ perform queries that don't map cleanly to models, or directly execute
In these cases, you can always access the database directly, routing around
the model layer entirely.
The object ``django.db.connection`` represents the
default database connection, and ``django.db.transaction`` represents the
default database transaction. To use the database connection, call
``connection.cursor()`` to get a cursor object. Then, call
``cursor.execute(sql, [params])`` to execute the SQL and ``cursor.fetchone()``
or ``cursor.fetchall()`` to return the resulting rows. After performing a data
changing operation, you should then call
``transaction.commit_unless_managed()`` to ensure your changes are committed
to the database. If your query is purely a data retrieval operation, no commit
is required. For example::
The object ``django.db.connection`` represents the default database
connection. To use the database connection, call ``connection.cursor()`` to
get a cursor object. Then, call ``cursor.execute(sql, [params])`` to execute
the SQL and ``cursor.fetchone()`` or ``cursor.fetchall()`` to return the
resulting rows.
For example::
from django.db import connection
def my_custom_sql():
from django.db import connection, transaction
cursor = connection.cursor()
# Data modifying operation - commit required
cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz])
transaction.commit_unless_managed()
# Data retrieval operation - no commit required
cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz])
row = cursor.fetchone()
return row
.. versionchanged:: 1.6
In Django 1.5 and earlier, after performing a data changing operation, you
had to call ``transaction.commit_unless_managed()`` to ensure your changes
were committed to the database. Since Django now defaults to database-level
autocommit, this isn't necessary any longer.
If you are using :doc:`more than one database </topics/db/multi-db>`, you can
use ``django.db.connections`` to obtain the connection (and cursor) for a
specific database. ``django.db.connections`` is a dictionary-like
@ -235,7 +236,6 @@ alias::
from django.db import connections
cursor = connections['my_db_alias'].cursor()
# Your code here...
transaction.commit_unless_managed(using='my_db_alias')
By default, the Python DB API will return results without their field
names, which means you end up with a ``list`` of values, rather than a
@ -260,27 +260,18 @@ Here is an example of the difference between the two::
>>> dictfetchall(cursor)
[{'parent_id': None, 'id': 54360982L}, {'parent_id': None, 'id': 54360880L}]
.. _transactions-and-raw-sql:
Transactions and raw SQL
------------------------
When you make a raw SQL call, Django will automatically mark the
current transaction as dirty. You must then ensure that the
transaction containing those calls is closed correctly. See :ref:`the
notes on the requirements of Django's transaction handling
<topics-db-transactions-requirements>` for more details.
Connections and cursors
-----------------------
``connection`` and ``cursor`` mostly implement the standard Python DB-API
described in :pep:`249` (except when it comes to :doc:`transaction handling
</topics/db/transactions>`). If you're not familiar with the Python DB-API, note
that the SQL statement in ``cursor.execute()`` uses placeholders, ``"%s"``,
rather than adding parameters directly within the SQL. If you use this
technique, the underlying database library will automatically add quotes and
escaping to your parameter(s) as necessary. (Also note that Django expects the
``"%s"`` placeholder, *not* the ``"?"`` placeholder, which is used by the SQLite
Python bindings. This is for the sake of consistency and sanity.)
described in :pep:`249` — except when it comes to :doc:`transaction handling
</topics/db/transactions>`.
If you're not familiar with the Python DB-API, note that the SQL statement in
``cursor.execute()`` uses placeholders, ``"%s"``, rather than adding
parameters directly within the SQL. If you use this technique, the underlying
database library will automatically escape your parameters as necessary.
Also note that Django expects the ``"%s"`` placeholder, *not* the ``"?"``
placeholder, which is used by the SQLite Python bindings. This is for the sake
of consistency and sanity.

View File

@ -4,21 +4,24 @@ Managing database transactions
.. module:: django.db.transaction
Django gives you a few ways to control how database transactions are managed,
if you're using a database that supports transactions.
Django gives you a few ways to control how database transactions are managed.
Django's default transaction behavior
=====================================
Django's default behavior is to run with an open transaction which it
commits automatically when any built-in, data-altering model function is
called. For example, if you call ``model.save()`` or ``model.delete()``, the
change will be committed immediately.
Django's default behavior is to run in autocommit mode. Each query is
immediately committed to the database. :ref:`See below for details
<autocommit-details>`.
This is much like the auto-commit setting for most databases. As soon as you
perform an action that needs to write to the database, Django produces the
``INSERT``/``UPDATE``/``DELETE`` statements and then does the ``COMMIT``.
There's no implicit ``ROLLBACK``.
..
Django uses transactions or savepoints automatically to guarantee the
integrity of ORM operations that require multiple queries, especially
:ref:`delete() <topics-db-queries-delete>` and :ref:`update()
<topics-db-queries-update>` queries.
.. versionchanged:: 1.6
Previous version of Django featured :ref:`a more complicated default
behavior <transactions-changes-from-1.5>`.
Tying transactions to HTTP requests
===================================
@ -26,7 +29,7 @@ Tying transactions to HTTP requests
The recommended way to handle transactions in Web requests is to tie them to
the request and response phases via Django's ``TransactionMiddleware``.
It works like this: When a request starts, Django starts a transaction. If the
It works like this. When a request starts, Django starts a transaction. If the
response is produced without problems, Django commits any pending transactions.
If the view function produces an exception, Django rolls back any pending
transactions.
@ -47,11 +50,11 @@ view functions, but also for all middleware modules that come after it. So if
you use the session middleware after the transaction middleware, session
creation will be part of the transaction.
The various cache middlewares are an exception:
``CacheMiddleware``, :class:`~django.middleware.cache.UpdateCacheMiddleware`,
and :class:`~django.middleware.cache.FetchFromCacheMiddleware` are never
affected. Even when using database caching, Django's cache backend uses its own
database cursor (which is mapped to its own database connection internally).
The various cache middlewares are an exception: ``CacheMiddleware``,
:class:`~django.middleware.cache.UpdateCacheMiddleware`, and
:class:`~django.middleware.cache.FetchFromCacheMiddleware` are never affected.
Even when using database caching, Django's cache backend uses its own database
connection internally.
.. note::
@ -116,7 +119,7 @@ managers, too.
.. function:: autocommit
Use the ``autocommit`` decorator to switch a view function to Django's
default commit behavior, regardless of the global transaction setting.
default commit behavior.
Example::
@ -195,14 +198,14 @@ managers, too.
Requirements for transaction handling
=====================================
Django requires that every transaction that is opened is closed before
the completion of a request. If you are using :func:`autocommit` (the
default commit mode) or :func:`commit_on_success`, this will be done
for you automatically (with the exception of :ref:`executing custom SQL
<executing-custom-sql>`). However, if you are manually managing
transactions (using the :func:`commit_manually` decorator), you must
ensure that the transaction is either committed or rolled back before
a request is completed.
Django requires that every transaction that is opened is closed before the
completion of a request.
If you are using :func:`autocommit` (the default commit mode) or
:func:`commit_on_success`, this will be done for you automatically. However,
if you are manually managing transactions (using the :func:`commit_manually`
decorator), you must ensure that the transaction is either committed or rolled
back before a request is completed.
This applies to all database operations, not just write operations. Even
if your transaction only reads from the database, the transaction must
@ -231,17 +234,17 @@ How to globally deactivate transaction management
=================================================
Control freaks can totally disable all transaction management by setting
:setting:`TRANSACTIONS_MANAGED` to ``True`` in the Django settings file.
:setting:`TRANSACTIONS_MANAGED` to ``True`` in the Django settings file. If
you do this, Django won't enable autocommit. You'll get the regular behavior
of the underlying database library.
If you do this, Django won't provide any automatic transaction management
whatsoever. Middleware will no longer implicitly commit transactions, and
you'll need to roll management yourself. This even requires you to commit
changes done by middleware somewhere else.
This requires you to commit explicitly every transaction, even those started
by Django or by third-party libraries. Thus, this is best used in situations
where you want to run your own transaction-controlling middleware or do
something really strange.
Thus, this is best used in situations where you want to run your own
transaction-controlling middleware or do something really strange. In almost
all situations, you'll be better off using the default behavior, or the
transaction middleware, and only modify selected functions as needed.
In almost all situations, you'll be better off using the default behavior, or
the transaction middleware, and only modify selected functions as needed.
.. _topics-db-transactions-savepoints:
@ -308,8 +311,11 @@ The following example demonstrates the use of savepoints::
transaction.commit()
Database-specific notes
=======================
Transactions in MySQL
=====================
---------------------
If you're using MySQL, your tables may or may not support transactions; it
depends on your MySQL version and the table types you're using. (By
@ -318,14 +324,14 @@ peculiarities are outside the scope of this article, but the MySQL site has
`information on MySQL transactions`_.
If your MySQL setup does *not* support transactions, then Django will function
in auto-commit mode: Statements will be executed and committed as soon as
in autocommit mode: Statements will be executed and committed as soon as
they're called. If your MySQL setup *does* support transactions, Django will
handle transactions as explained in this document.
.. _information on MySQL transactions: http://dev.mysql.com/doc/refman/5.0/en/sql-syntax-transactions.html
Handling exceptions within PostgreSQL transactions
==================================================
--------------------------------------------------
When a call to a PostgreSQL cursor raises an exception (typically
``IntegrityError``), all subsequent SQL in the same transaction will fail with
@ -338,7 +344,7 @@ force_insert/force_update flag, or invoking custom SQL.
There are several ways to recover from this sort of error.
Transaction rollback
--------------------
~~~~~~~~~~~~~~~~~~~~
The first option is to roll back the entire transaction. For example::
@ -355,7 +361,7 @@ made by ``a.save()`` would be lost, even though that operation raised no error
itself.
Savepoint rollback
------------------
~~~~~~~~~~~~~~~~~~
If you are using PostgreSQL 8 or later, you can use :ref:`savepoints
<topics-db-transactions-savepoints>` to control the extent of a rollback.
@ -375,25 +381,135 @@ offending operation, rather than the entire transaction. For example::
In this example, ``a.save()`` will not be undone in the case where
``b.save()`` raises an exception.
Database-level autocommit
-------------------------
Under the hood
==============
With PostgreSQL 8.2 or later, there is an advanced option to run PostgreSQL
with :doc:`database-level autocommit </ref/databases>`. If you use this option,
there is no constantly open transaction, so it is always possible to continue
after catching an exception. For example::
.. _autocommit-details:
a.save() # succeeds
try:
b.save() # Could throw exception
except IntegrityError:
pass
c.save() # succeeds
Details on autocommit
---------------------
.. note::
In the SQL standards, each SQL query starts a transaction, unless one is
already in progress. Such transactions must then be committed or rolled back.
This is not the same as the :ref:`autocommit decorator
<topics-db-transactions-autocommit>`. When using database level autocommit
there is no database transaction at all. The ``autocommit`` decorator
still uses transactions, automatically committing each transaction when
a database modifying operation occurs.
This isn't always convenient for application developers. To alleviate this
problem, most databases provide an autocommit mode. When autocommit is turned
on, each SQL query is wrapped in its own transaction. In other words, the
transaction is not only automatically started, but also automatically
committed.
:pep:`249`, the Python Database API Specification v2.0, requires autocommit to
be initially turned off. Django overrides this default and turns autocommit
on.
To avoid this, you can :ref:`deactivate the transaction management
<deactivate-transaction-management>`, but it isn't recommended.
.. versionchanged:: 1.6
Before Django 1.6, autocommit was turned off, and it was emulated by
forcing a commit after write operations in the ORM.
.. warning::
If you're using the database API directly — for instance, you're running
SQL queries with ``cursor.execute()`` — be aware that autocommit is on,
and consider wrapping your operations in a transaction to ensure
consistency.
.. _transaction-states:
Transaction management states
-----------------------------
At any time, each database connection is in one of these two states:
- **auto mode**: autocommit is enabled;
- **managed mode**: autocommit is disabled.
Django starts in auto mode. ``TransactionMiddleware``,
:func:`commit_on_success` and :func:`commit_manually` activate managed mode;
:func:`autocommit` activates auto mode.
Internally, Django keeps a stack of states. Activations and deactivations must
be balanced.
For example, at the beginning of each HTTP request, ``TransactionMiddleware``
switches to managed mode; at the end of the request, it commits or rollbacks,
and switches back to auto mode.
.. admonition:: Nesting decorators / context managers
:func:`commit_on_success` has two effects: it changes the transaction
state, and defines an atomic transaction block.
Nesting with :func:`autocommit` and :func:`commit_manually` will give the
expected results in terms of transaction state, but not in terms of
transaction semantics. Most often, the inner block will commit, breaking
the atomicity of the outer block.
Django currently doesn't provide any APIs to create transactions in auto mode.
.. _transactions-changes-from-1.5:
Changes from Django 1.5 and earlier
===================================
Since version 1.6, Django uses database-level autocommit in auto mode.
Previously, it implemented application-level autocommit by triggering a commit
after each ORM write.
As a consequence, each database query (for instance, an
ORM read) started a transaction that lasted until the next ORM write. Such
"automatic transactions" no longer exist in Django 1.6.
There are four known scenarios where this is backwards-incompatible.
Note that managed mode isn't affected at all. This section assumes auto mode.
See the :ref:`description of modes <transaction-states>` above.
Sequences of custom SQL queries
-------------------------------
If you're executing several :ref:`custom SQL queries <executing-custom-sql>`
in a row, each one now runs in its own transaction, instead of sharing the
same "automatic transaction". If you need to enforce atomicity, you must wrap
the sequence of queries in :func:`commit_on_success`.
To check for this problem, look for calls to ``cursor.execute()``. They're
usually followed by a call to ``transaction.commit_unless_managed``, which
isn't necessary any more and should be removed.
Select for update
-----------------
If you were relying on "automatic transactions" to provide locking between
:meth:`~django.db.models.query.QuerySet.select_for_update` and a subsequent
write operation — an extremely fragile design, but nonetheless possible — you
must wrap the relevant code in :func:`commit_on_success`.
Using a high isolation level
----------------------------
If you were using the "repeatable read" isolation level or higher, and if you
relied on "automatic transactions" to guarantee consistency between successive
reads, the new behavior is backwards-incompatible. To maintain consistency,
you must wrap such sequences in :func:`commit_on_success`.
MySQL defaults to "repeatable read" and SQLite to "serializable"; they may be
affected by this problem.
At the "read committed" isolation level or lower, "automatic transactions"
have no effect on the semantics of any sequence of ORM operations.
PostgreSQL and Oracle default to "read committed" and aren't affected, unless
you changed the isolation level.
Using unsupported database features
-----------------------------------
With triggers, views, or functions, it's possible to make ORM reads result in
database modifications. Django 1.5 and earlier doesn't deal with this case and
it's theoretically possible to observe a different behavior after upgrading to
Django 1.6 or later. In doubt, use :func:`commit_on_success` to enforce
integrity.