Enabled database-level autocommit for all backends.

This is mostly a documentation change.

It has the same backwards-incompatibility consequences as those
described for PostgreSQL in a previous commit.

django/db/backends/__init__.py

@@ -98,6 +98,8 @@ class BaseDatabaseWrapper(object):
         conn_params = self.get_connection_params()
         self.connection = self.get_new_connection(conn_params)
         self.init_connection_state()
+        if not settings.TRANSACTIONS_MANAGED:
+            self.set_autocommit()
         connection_created.send(sender=self.__class__, connection=self)
 
     def ensure_connection(self):

django/db/backends/postgresql_psycopg2/base.py

@@ -136,7 +136,6 @@ class DatabaseWrapper(BaseDatabaseWrapper):
                 self.connection.cursor().execute(
                         self.ops.set_time_zone_sql(), [tz])
         self.connection.set_isolation_level(self.isolation_level)
-        self.set_autocommit(not settings.TRANSACTIONS_MANAGED)
 
     def create_cursor(self):
         cursor = self.connection.cursor()

docs/ref/databases.txt

@@ -69,7 +69,6 @@ even ``0``, because it doesn't make sense to maintain a connection that's
 unlikely to be reused. This will help keep the number of simultaneous
 connections to this database small.
 
-
 The development server creates a new thread for each request it handles,
 negating the effect of persistent connections.
 
@@ -104,7 +103,8 @@ Optimizing PostgreSQL's configuration
 Django needs the following parameters for its database connections:
 
 - ``client_encoding``: ``'UTF8'``,
-- ``default_transaction_isolation``: ``'read committed'``,
+- ``default_transaction_isolation``: ``'read committed'`` by default,
+  or the value set in the connection options (see below),
 - ``timezone``: ``'UTC'`` when :setting:`USE_TZ` is ``True``, value of
   :setting:`TIME_ZONE` otherwise.
 
@@ -118,30 +118,16 @@ will do some additional queries to set these parameters.
 
 .. _ALTER ROLE: http://www.postgresql.org/docs/current/interactive/sql-alterrole.html
 
-Transaction handling
----------------------
-
-:doc:`By default </topics/db/transactions>`, Django runs with an open
-transaction which it commits automatically when any built-in, data-altering
-model function is called. The PostgreSQL backends normally operate the same as
-any other Django backend in this respect.
-
 .. _postgresql-autocommit-mode:
 
 Autocommit mode
-~~~~~~~~~~~~~~~
+---------------
 
-If your application is particularly read-heavy and doesn't make many
+.. versionchanged:: 1.6
-database writes, the overhead of a constantly open transaction can
+
-sometimes be noticeable. For those situations, you can configure Django
+In previous versions of Django, database-level autocommit could be enabled by
-to use *"autocommit"* behavior for the connection, meaning that each database
+setting the ``autocommit`` key in the :setting:`OPTIONS` part of your database
-operation will normally be in its own transaction, rather than having
+configuration in :setting:`DATABASES`::
-the transaction extend over multiple operations. In this case, you can
-still manually start a transaction if you're doing something that
-requires consistency across multiple database operations. The
-autocommit behavior is enabled by setting the ``autocommit`` key in
-the :setting:`OPTIONS` part of your database configuration in
-:setting:`DATABASES`::
 
     DATABASES = {
         # ...
@@ -150,29 +136,11 @@ the :setting:`OPTIONS` part of your database configuration in
         },
     }
 
-In this configuration, Django still ensures that :ref:`delete()
+Since Django 1.6, autocommit is turned on by default. This configuration is
-<topics-db-queries-delete>` and :ref:`update() <topics-db-queries-update>`
+ignored and can be safely removed.
-queries run inside a single transaction, so that either all the affected
-objects are changed or none of them are.
-
-.. admonition:: This is database-level autocommit
-
-    This functionality is not the same as the :ref:`autocommit
-    <topics-db-transactions-autocommit>` decorator. That decorator is
-    a Django-level implementation that commits automatically after
-    data changing operations. The feature enabled using the
-    :setting:`OPTIONS` option provides autocommit behavior at the
-    database adapter level. It commits after *every* operation.
-
-If you are using this feature and performing an operation akin to delete or
-updating that requires multiple operations, you are strongly recommended to
-wrap you operations in manual transaction handling to ensure data consistency.
-You should also audit your existing code for any instances of this behavior
-before enabling this feature. It's faster, but it provides less automatic
-protection for multi-call operations.
 
 Isolation level
-~~~~~~~~~~~~~~~
+---------------
 
 .. versionadded:: 1.6
 
@@ -200,7 +168,7 @@ such as ``REPEATABLE READ`` or ``SERIALIZABLE``, set it in the
 .. _postgresql-isolation-levels: http://www.postgresql.org/docs/devel/static/transaction-iso.html
 
 Indexes for ``varchar`` and ``text`` columns
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+--------------------------------------------
 
 When specifying ``db_index=True`` on your model fields, Django typically
 outputs a single ``CREATE INDEX`` statement.  However, if the database type
@@ -457,7 +425,7 @@ Savepoints
 Both the Django ORM and MySQL (when using the InnoDB :ref:`storage engine
 <mysql-storage-engines>`) support database :ref:`savepoints
 <topics-db-transactions-savepoints>`, but this feature wasn't available in
-Django until version 1.4 when such supports was added.
+Django until version 1.4 when such support was added.
 
 If you use the MyISAM storage engine please be aware of the fact that you will
 receive database-generated errors if you try to use the :ref:`savepoint-related

docs/ref/request-response.txt

@@ -814,8 +814,8 @@ generating large CSV files.
 .. admonition:: Performance considerations
 
     Django is designed for short-lived requests. Streaming responses will tie
-    a worker process and keep a database connection idle in transaction for
+    a worker process for the entire duration of the response. This may result
-    the entire duration of the response. This may result in poor performance.
+    in poor performance.
 
     Generally speaking, you should perform expensive tasks outside of the
     request-response cycle, rather than resorting to a streamed response.

docs/releases/1.6.txt

@@ -30,6 +30,18 @@ prevention <clickjacking-prevention>` are turned on.
 If the default templates don't suit your tastes, you can use :ref:`custom
 project and app templates <custom-app-and-project-templates>`.
 
+Improved transaction management
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Django's transaction management was overhauled. Database-level autocommit is
+now turned on by default. This makes transaction handling more explicit and
+should improve performance. The existing APIs were deprecated, and new APIs
+were introduced, as described in :doc:`/topics/db/transactions`.
+
+Please review carefully the list of :ref:`known backwards-incompatibilities
+<transactions-changes-from-1.5>` to determine if you need to make changes in
+your code.
+
 Persistent database connections
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -148,6 +160,16 @@ Backwards incompatible changes in 1.6
     deprecation timeline for a given feature, its removal may appear as a
     backwards incompatible change.
 
+* Database-level autocommit is enabled by default in Django 1.6. While this
+  doesn't change the general spirit of Django's transaction management, there
+  are a few known backwards-incompatibities, described in the :ref:`transaction
+  management docs <transactions-changes-from-1.5>`. You should review your code
+  to determine if you're affected.
+
+* In previous versions, database-level autocommit was only an option for
+  PostgreSQL, and it was disabled by default. This option is now
+  :ref:`ignored <postgresql-autocommit-mode>`.
+
 * The ``django.db.models.query.EmptyQuerySet`` can't be instantiated any more -
   it is only usable as a marker class for checking if
   :meth:`~django.db.models.query.QuerySet.none` has been called:

docs/topics/db/sql.txt

@@ -201,31 +201,32 @@ perform queries that don't map cleanly to models, or directly execute
 In these cases, you can always access the database directly, routing around
 the model layer entirely.
 
-The object ``django.db.connection`` represents the
+The object ``django.db.connection`` represents the default database
-default database connection, and ``django.db.transaction`` represents the
+connection. To use the database connection, call ``connection.cursor()`` to
-default database transaction. To use the database connection, call
+get a cursor object. Then, call ``cursor.execute(sql, [params])`` to execute
-``connection.cursor()`` to get a cursor object. Then, call
+the SQL and ``cursor.fetchone()`` or ``cursor.fetchall()`` to return the
-``cursor.execute(sql, [params])`` to execute the SQL and ``cursor.fetchone()``
+resulting rows.
-or ``cursor.fetchall()`` to return the resulting rows. After performing a data
+
-changing operation, you should then call
+For example::
-``transaction.commit_unless_managed()`` to ensure your changes are committed
+
-to the database. If your query is purely a data retrieval operation, no commit
+    from django.db import connection
-is required. For example::
 
     def my_custom_sql():
-        from django.db import connection, transaction
         cursor = connection.cursor()
 
-        # Data modifying operation - commit required
         cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz])
-        transaction.commit_unless_managed()
 
-        # Data retrieval operation - no commit required
         cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz])
         row = cursor.fetchone()
 
         return row
 
+.. versionchanged:: 1.6
+    In Django 1.5 and earlier, after performing a data changing operation, you
+    had to call ``transaction.commit_unless_managed()`` to ensure your changes
+    were committed to the database. Since Django now defaults to database-level
+    autocommit, this isn't necessary any longer.
+
 If you are using :doc:`more than one database </topics/db/multi-db>`, you can
 use ``django.db.connections`` to obtain the connection (and cursor) for a
 specific database. ``django.db.connections`` is a dictionary-like
@@ -235,7 +236,6 @@ alias::
     from django.db import connections
     cursor = connections['my_db_alias'].cursor()
     # Your code here...
-    transaction.commit_unless_managed(using='my_db_alias')
 
 By default, the Python DB API will return results without their field
 names, which means you end up with a ``list`` of values, rather than a
@@ -260,27 +260,18 @@ Here is an example of the difference between the two::
     >>> dictfetchall(cursor)
     [{'parent_id': None, 'id': 54360982L}, {'parent_id': None, 'id': 54360880L}]
 
-
-.. _transactions-and-raw-sql:
-
-Transactions and raw SQL
-------------------------
-
-When you make a raw SQL call, Django will automatically mark the
-current transaction as dirty. You must then ensure that the
-transaction containing those calls is closed correctly. See :ref:`the
-notes on the requirements of Django's transaction handling
-<topics-db-transactions-requirements>` for more details.
-
 Connections and cursors
 -----------------------
 
 ``connection`` and ``cursor`` mostly implement the standard Python DB-API
-described in :pep:`249` (except when it comes to :doc:`transaction handling
+described in :pep:`249` — except when it comes to :doc:`transaction handling
-</topics/db/transactions>`). If you're not familiar with the Python DB-API, note
+</topics/db/transactions>`.
-that the SQL statement in ``cursor.execute()`` uses placeholders, ``"%s"``,
+
-rather than adding parameters directly within the SQL. If you use this
+If you're not familiar with the Python DB-API, note that the SQL statement in
-technique, the underlying database library will automatically add quotes and
+``cursor.execute()`` uses placeholders, ``"%s"``, rather than adding
-escaping to your parameter(s) as necessary. (Also note that Django expects the
+parameters directly within the SQL. If you use this technique, the underlying
-``"%s"`` placeholder, *not* the ``"?"`` placeholder, which is used by the SQLite
+database library will automatically escape your parameters as necessary.
-Python bindings. This is for the sake of consistency and sanity.)
+
+Also note that Django expects the ``"%s"`` placeholder, *not* the ``"?"``
+placeholder, which is used by the SQLite Python bindings. This is for the sake
+of consistency and sanity.

docs/topics/db/transactions.txt

@@ -4,21 +4,24 @@ Managing database transactions
 
 .. module:: django.db.transaction
 
-Django gives you a few ways to control how database transactions are managed,
+Django gives you a few ways to control how database transactions are managed.
-if you're using a database that supports transactions.
 
 Django's default transaction behavior
 =====================================
 
-Django's default behavior is to run with an open transaction which it
+Django's default behavior is to run in autocommit mode. Each query is
-commits automatically when any built-in, data-altering model function is
+immediately committed to the database. :ref:`See below for details
-called. For example, if you call ``model.save()`` or ``model.delete()``, the
+<autocommit-details>`.
-change will be committed immediately.
 
-This is much like the auto-commit setting for most databases. As soon as you
+..
-perform an action that needs to write to the database, Django produces the
+   Django uses transactions or savepoints automatically to guarantee the
-``INSERT``/``UPDATE``/``DELETE`` statements and then does the ``COMMIT``.
+   integrity of ORM operations that require multiple queries, especially
-There's no implicit ``ROLLBACK``.
+   :ref:`delete() <topics-db-queries-delete>` and :ref:`update()
+   <topics-db-queries-update>` queries.
+
+.. versionchanged:: 1.6
+    Previous version of Django featured :ref:`a more complicated default
+    behavior <transactions-changes-from-1.5>`.
 
 Tying transactions to HTTP requests
 ===================================
@@ -26,7 +29,7 @@ Tying transactions to HTTP requests
 The recommended way to handle transactions in Web requests is to tie them to
 the request and response phases via Django's ``TransactionMiddleware``.
 
-It works like this: When a request starts, Django starts a transaction. If the
+It works like this. When a request starts, Django starts a transaction. If the
 response is produced without problems, Django commits any pending transactions.
 If the view function produces an exception, Django rolls back any pending
 transactions.
@@ -47,11 +50,11 @@ view functions, but also for all middleware modules that come after it. So if
 you use the session middleware after the transaction middleware, session
 creation will be part of the transaction.
 
-The various cache middlewares are an exception:
+The various cache middlewares are an exception: ``CacheMiddleware``,
-``CacheMiddleware``, :class:`~django.middleware.cache.UpdateCacheMiddleware`,
+:class:`~django.middleware.cache.UpdateCacheMiddleware`, and
-and :class:`~django.middleware.cache.FetchFromCacheMiddleware` are never
+:class:`~django.middleware.cache.FetchFromCacheMiddleware` are never affected.
-affected. Even when using database caching, Django's cache backend uses its own
+Even when using database caching, Django's cache backend uses its own database
-database cursor (which is mapped to its own database connection internally).
+connection internally.
 
 .. note::
 
@@ -116,7 +119,7 @@ managers, too.
 .. function:: autocommit
 
     Use the ``autocommit`` decorator to switch a view function to Django's
-    default commit behavior, regardless of the global transaction setting.
+    default commit behavior.
 
     Example::
 
@@ -195,14 +198,14 @@ managers, too.
 Requirements for transaction handling
 =====================================
 
-Django requires that every transaction that is opened is closed before
+Django requires that every transaction that is opened is closed before the
-the completion of a request. If you are using :func:`autocommit` (the
+completion of a request.
-default commit mode) or :func:`commit_on_success`, this will be done
+
-for you automatically (with the exception of :ref:`executing custom SQL
+If you are using :func:`autocommit` (the default commit mode) or
-<executing-custom-sql>`). However, if you are manually managing
+:func:`commit_on_success`, this will be done for you automatically. However,
-transactions (using the :func:`commit_manually` decorator), you must
+if you are manually managing transactions (using the :func:`commit_manually`
-ensure that the transaction is either committed or rolled back before
+decorator), you must ensure that the transaction is either committed or rolled
-a request is completed.
+back before a request is completed.
 
 This applies to all database operations, not just write operations. Even
 if your transaction only reads from the database, the transaction must
@@ -231,17 +234,17 @@ How to globally deactivate transaction management
 =================================================
 
 Control freaks can totally disable all transaction management by setting
-:setting:`TRANSACTIONS_MANAGED` to ``True`` in the Django settings file.
+:setting:`TRANSACTIONS_MANAGED` to ``True`` in the Django settings file. If
+you do this, Django won't enable autocommit. You'll get the regular behavior
+of the underlying database library.
 
-If you do this, Django won't provide any automatic transaction management
+This requires you to commit explicitly every transaction, even those started
-whatsoever. Middleware will no longer implicitly commit transactions, and
+by Django or by third-party libraries. Thus, this is best used in situations
-you'll need to roll management yourself. This even requires you to commit
+where you want to run your own transaction-controlling middleware or do
-changes done by middleware somewhere else.
+something really strange.
 
-Thus, this is best used in situations where you want to run your own
+In almost all situations, you'll be better off using the default behavior, or
-transaction-controlling middleware or do something really strange. In almost
+the transaction middleware, and only modify selected functions as needed.
-all situations, you'll be better off using the default behavior, or the
-transaction middleware, and only modify selected functions as needed.
 
 .. _topics-db-transactions-savepoints:
 
@@ -308,8 +311,11 @@ The following example demonstrates the use of savepoints::
 
       transaction.commit()
 
+Database-specific notes
+=======================
+
 Transactions in MySQL
-=====================
+---------------------
 
 If you're using MySQL, your tables may or may not support transactions; it
 depends on your MySQL version and the table types you're using. (By
@@ -318,14 +324,14 @@ peculiarities are outside the scope of this article, but the MySQL site has
 `information on MySQL transactions`_.
 
 If your MySQL setup does *not* support transactions, then Django will function
-in auto-commit mode: Statements will be executed and committed as soon as
+in autocommit mode: Statements will be executed and committed as soon as
 they're called. If your MySQL setup *does* support transactions, Django will
 handle transactions as explained in this document.
 
 .. _information on MySQL transactions: http://dev.mysql.com/doc/refman/5.0/en/sql-syntax-transactions.html
 
 Handling exceptions within PostgreSQL transactions
-==================================================
+--------------------------------------------------
 
 When a call to a PostgreSQL cursor raises an exception (typically
 ``IntegrityError``), all subsequent SQL in the same transaction will fail with
@@ -338,7 +344,7 @@ force_insert/force_update flag, or invoking custom SQL.
 There are several ways to recover from this sort of error.
 
 Transaction rollback
---------------------
+~~~~~~~~~~~~~~~~~~~~
 
 The first option is to roll back the entire transaction. For example::
 
@@ -355,7 +361,7 @@ made by ``a.save()`` would be lost, even though that operation raised no error
 itself.
 
 Savepoint rollback
-------------------
+~~~~~~~~~~~~~~~~~~
 
 If you are using PostgreSQL 8 or later, you can use :ref:`savepoints
 <topics-db-transactions-savepoints>` to control the extent of a rollback.
@@ -375,25 +381,135 @@ offending operation, rather than the entire transaction. For example::
 In this example, ``a.save()`` will not be undone in the case where
 ``b.save()`` raises an exception.
 
-Database-level autocommit
+Under the hood
--------------------------
+==============
 
-With PostgreSQL 8.2 or later, there is an advanced option to run PostgreSQL
+.. _autocommit-details:
-with :doc:`database-level autocommit </ref/databases>`. If you use this option,
-there is no constantly open transaction, so it is always possible to continue
-after catching an exception. For example::
 
-    a.save() # succeeds
+Details on autocommit
-    try:
+---------------------
-        b.save() # Could throw exception
-    except IntegrityError:
-        pass
-    c.save() # succeeds
 
-.. note::
+In the SQL standards, each SQL query starts a transaction, unless one is
+already in progress. Such transactions must then be committed or rolled back.
 
-    This is not the same as the :ref:`autocommit decorator
+This isn't always convenient for application developers. To alleviate this
-    <topics-db-transactions-autocommit>`. When using database level autocommit
+problem, most databases provide an autocommit mode. When autocommit is turned
-    there is no database transaction at all. The ``autocommit`` decorator
+on, each SQL query is wrapped in its own transaction. In other words, the
-    still uses transactions, automatically committing each transaction when
+transaction is not only automatically started, but also automatically
-    a database modifying operation occurs.
+committed.
+
+:pep:`249`, the Python Database API Specification v2.0, requires autocommit to
+be initially turned off. Django overrides this default and turns autocommit
+on.
+
+To avoid this, you can :ref:`deactivate the transaction management
+<deactivate-transaction-management>`, but it isn't recommended.
+
+.. versionchanged:: 1.6
+    Before Django 1.6, autocommit was turned off, and it was emulated by
+    forcing a commit after write operations in the ORM.
+
+.. warning::
+
+    If you're using the database API directly — for instance, you're running
+    SQL queries with ``cursor.execute()`` — be aware that autocommit is on,
+    and consider wrapping your operations in a transaction to ensure
+    consistency.
+
+.. _transaction-states:
+
+Transaction management states
+-----------------------------
+
+At any time, each database connection is in one of these two states:
+
+- **auto mode**: autocommit is enabled;
+- **managed mode**: autocommit is disabled.
+
+Django starts in auto mode. ``TransactionMiddleware``,
+:func:`commit_on_success` and :func:`commit_manually` activate managed mode;
+:func:`autocommit` activates auto mode.
+
+Internally, Django keeps a stack of states. Activations and deactivations must
+be balanced.
+
+For example, at the beginning of each HTTP request, ``TransactionMiddleware``
+switches to managed mode; at the end of the request, it commits or rollbacks,
+and switches back to auto mode.
+
+.. admonition:: Nesting decorators / context managers
+
+    :func:`commit_on_success` has two effects: it changes the transaction
+    state, and defines an atomic transaction block.
+
+    Nesting with :func:`autocommit` and :func:`commit_manually` will give the
+    expected results in terms of transaction state, but not in terms of
+    transaction semantics. Most often, the inner block will commit, breaking
+    the atomicity of the outer block.
+
+Django currently doesn't provide any APIs to create transactions in auto mode.
+
+.. _transactions-changes-from-1.5:
+
+Changes from Django 1.5 and earlier
+===================================
+
+Since version 1.6, Django uses database-level autocommit in auto mode.
+
+Previously, it implemented application-level autocommit by triggering a commit
+after each ORM write.
+
+As a consequence, each database query (for instance, an
+ORM read) started a transaction that lasted until the next ORM write. Such
+"automatic transactions" no longer exist in Django 1.6.
+
+There are four known scenarios where this is backwards-incompatible.
+
+Note that managed mode isn't affected at all. This section assumes auto mode.
+See the :ref:`description of modes <transaction-states>` above.
+
+Sequences of custom SQL queries
+-------------------------------
+
+If you're executing several :ref:`custom SQL queries <executing-custom-sql>`
+in a row, each one now runs in its own transaction, instead of sharing the
+same "automatic transaction". If you need to enforce atomicity, you must wrap
+the sequence of queries in :func:`commit_on_success`.
+
+To check for this problem, look for calls to ``cursor.execute()``. They're
+usually followed by a call to ``transaction.commit_unless_managed``, which
+isn't necessary any more and should be removed.
+
+Select for update
+-----------------
+
+If you were relying on "automatic transactions" to provide locking between
+:meth:`~django.db.models.query.QuerySet.select_for_update` and a subsequent
+write operation — an extremely fragile design, but nonetheless possible — you
+must wrap the relevant code in :func:`commit_on_success`.
+
+Using a high isolation level
+----------------------------
+
+If you were using the "repeatable read" isolation level or higher, and if you
+relied on "automatic transactions" to guarantee consistency between successive
+reads, the new behavior is backwards-incompatible. To maintain consistency,
+you must wrap such sequences in :func:`commit_on_success`.
+
+MySQL defaults to "repeatable read" and SQLite to "serializable"; they may be
+affected by this problem.
+
+At the "read committed" isolation level or lower, "automatic transactions"
+have no effect on the semantics of any sequence of ORM operations.
+
+PostgreSQL and Oracle default to "read committed" and aren't affected, unless
+you changed the isolation level.
+
+Using unsupported database features
+-----------------------------------
+
+With triggers, views, or functions, it's possible to make ORM reads result in
+database modifications. Django 1.5 and earlier doesn't deal with this case and
+it's theoretically possible to observe a different behavior after upgrading to
+Django 1.6 or later. In doubt, use :func:`commit_on_success` to enforce
+integrity.