From 79cc66692ad282b41a15fba9da9222c3140d4405 Mon Sep 17 00:00:00 2001 From: evildmp Date: Fri, 6 Sep 2013 12:57:25 -0500 Subject: [PATCH] Consolidated documentation for F() and Q() --- docs/index.txt | 3 +- docs/ref/models/fields.txt | 11 +-- docs/ref/models/index.txt | 1 + docs/ref/models/instances.txt | 34 +++------ docs/ref/models/queries.txt | 127 ++++++++++++++++++++++++++++++++ docs/releases/1.1-alpha-1.txt | 6 +- docs/releases/1.1.txt | 2 +- docs/releases/1.3.txt | 2 +- docs/releases/1.5-alpha-1.txt | 11 +-- docs/releases/1.5-beta-1.txt | 11 +-- docs/releases/1.5.txt | 11 +-- docs/topics/db/optimization.txt | 2 +- docs/topics/db/queries.txt | 22 ++---- 13 files changed, 179 insertions(+), 64 deletions(-) create mode 100644 docs/ref/models/queries.txt diff --git a/docs/index.txt b/docs/index.txt index c58be5fcfc..6812d73206 100644 --- a/docs/index.txt +++ b/docs/index.txt @@ -65,7 +65,8 @@ manipulating the data of your Web application. Learn more about it below: * **QuerySets:** :doc:`Executing queries ` | - :doc:`QuerySet method reference ` + :doc:`QuerySet method reference ` | + :doc:`Query-related classes ` * **Model instances:** :doc:`Instance methods ` | diff --git a/docs/ref/models/fields.txt b/docs/ref/models/fields.txt index 194290581e..dd0d025ab2 100644 --- a/docs/ref/models/fields.txt +++ b/docs/ref/models/fields.txt @@ -1062,11 +1062,12 @@ define the details of how the relation works. only allows the choice of related objects with a ``pub_date`` before the current date to be chosen. - Instead of a dictionary this can also be a :class:`~django.db.models.Q` - object for more :ref:`complex queries `. However, - if ``limit_choices_to`` is a :class:`~django.db.models.Q` object then it - will only have an effect on the choices available in the admin when the - field is not listed in ``raw_id_fields`` in the ``ModelAdmin`` for the model. + Instead of a dictionary this can also be a :class:`Q object + ` for more :ref:`complex queries + `. However, if ``limit_choices_to`` is a :class:`Q + object ` then it will only have an effect on the + choices available in the admin when the field is not listed in + ``raw_id_fields`` in the ``ModelAdmin`` for the model. .. attribute:: ForeignKey.related_name diff --git a/docs/ref/models/index.txt b/docs/ref/models/index.txt index b5896c35ed..c61bb35a5f 100644 --- a/docs/ref/models/index.txt +++ b/docs/ref/models/index.txt @@ -12,3 +12,4 @@ Model API reference. For introductory material, see :doc:`/topics/db/models`. options instances querysets + queries diff --git a/docs/ref/models/instances.txt b/docs/ref/models/instances.txt index 3ad22fbb12..6bc6917a9e 100644 --- a/docs/ref/models/instances.txt +++ b/docs/ref/models/instances.txt @@ -342,6 +342,8 @@ only. Using ``update_fields`` will force an update similarly to ``force_update``. +.. _ref-models-field-updates-using-f-expressions: + Updating attributes based on existing fields -------------------------------------------- @@ -356,35 +358,21 @@ achieve this is to do something like:: If the old ``number_sold`` value retrieved from the database was 10, then the value of 11 will be written back to the database. -This sequence has a standard update problem in that it contains a race -condition. If another thread of execution has already saved an updated value -after the current thread retrieved the old value, the current thread will only -save the old value plus one, rather than the new (current) value plus one. - -The process can be made robust and slightly faster by expressing the update -relative to the original field value, rather than as an explicit assignment of -a new value. Django provides :ref:`F() expressions ` for -performing this kind of relative update. Using ``F()`` expressions, the -previous example is expressed as:: +The process can be made robust, :ref:`avoiding a race condition +`, as well as slightly faster by expressing +the update relative to the original field value, rather than as an explicit +assignment of a new value. Django provides :class:`F expressions +` for performing this kind of relative update. Using +:class:`F expressions `, the previous example is expressed +as:: >>> from django.db.models import F >>> product = Product.objects.get(name='Venezuelan Beaver Cheese') >>> product.number_sold = F('number_sold') + 1 >>> product.save() -This approach doesn't use the initial value from the database. Instead, it -makes the database do the update based on whatever value is current at the time -that the :meth:`~Model.save()` is executed. - -Once the object has been saved, you must reload the object in order to access -the actual value that was applied to the updated field:: - - >>> product = Products.objects.get(pk=product.pk) - >>> print(product.number_sold) - 42 - -For more details, see the documentation on :ref:`F() expressions -` and their :ref:`use in update queries +For more details, see the documentation on :class:`F expressions +` and their :ref:`use in update queries `. Specifying which fields to save diff --git a/docs/ref/models/queries.txt b/docs/ref/models/queries.txt new file mode 100644 index 0000000000..ebc5142b30 --- /dev/null +++ b/docs/ref/models/queries.txt @@ -0,0 +1,127 @@ +===================== +Query-related classes +===================== + +.. currentmodule:: django.db.models + +This document provides reference material for query-related tools not +documented elsewhere. + +``F()`` expressions +=================== + +.. class:: F + +An ``F()`` object represents the value of a model field. It makes it possible +to refer to model field values and perform database operations using them +without actually having to pull them out of the database into Python memory. + +Instead, Django uses the ``F()`` object to generate a SQL expression that +describes the required operation at the database level. + +This is easiest to understand though an example. Normally, one might do +something like this:: + + # Tintin filed a news story! + reporter = Reporters.objects.get(name='Tintin') + reporter.stories_filed += 1 + reporter.save() + +Here, we have pulled the value of ``reporter.stories_filed`` from the database +into memory and manipulated it using familiar Python operators, and then saved +the object back to the database. But instead we could also have done:: + + from django.db.models import F + reporter = Reporters.objects.get(name='Tintin') + reporter.stories_filed = F('stories_filed') + 1 + reporter.save() + +Although ``reporter.stories_filed = F('stories_filed') + 1`` looks like a +normal Python assignment of value to an instance attribute, in fact it's an SQL +construct describing an operation on the database. + +When Django encounters an instance of ``F()``, it overrides the standard Python +operators to create an encapsulated SQL expression; in this case, one which +instructs the database to increment the database field represented by +``reporter.stories_filed``. + +Whatever value is or was on ``reporter.stories_filed``, Python never gets to +know about it - it is dealt with entirely by the database. All Python does, +through Django's ``F()`` class, is create the SQL syntax to refer to the field +and describe the operation. + +.. note:: + + In order to access the new value that has been saved in this way, the object + will need to be reloaded:: + + reporter = Reporters.objects.get(pk=reporter.pk) + +As well as being used in operations on single instances as above, ``F()`` can +be used on ``QuerySets`` of object instances, with ``update()``. This reduces +the two queries we were using above - the ``get()`` and the +:meth:`~Model.save()` - to just one:: + + reporter = Reporters.objects.filter(name='Tintin') + reporter.update(stories_filed=F('stories_filed') + 1) + +We can also use :meth:`~django.db.models.query.QuerySet.update()` to increment +the field value on multiple objects - which could be very much faster than +pulling them all into Python from the database, looping over them, incrementing +the field value of each one, and saving each one back to the database:: + + Reporter.objects.all().update(stories_filed=F('stories_filed) + 1) + +``F()`` therefore can offer performance advantages by: + +* getting the database, rather than Python, to do work +* reducing the number of queries some operations require + +.. _avoiding-race-conditions-using-f: + +Avoiding race conditions using ``F()`` +-------------------------------------- + +Another useful benefit of ``F()`` is that having the database - rather than +Python - update a field's value avoids a *race condition*. + +If two Python threads execute the code in the first example above, one thread +could retrieve, increment, and save a field's value after the other has +retrieved it from the database. The value that the second thread saves will be +based on the original value; the work of the first thread will simply be lost. + +If the database is responsible for updating the field, the process is more +robust: it will only ever update the field based on the value of the field in +the database when the :meth:`~Model.save()` or ``update()`` is executed, rather +than based on its value when the instance was retrieved. + +Using ``F()`` in filters +------------------------ + +``F()`` is also very useful in ``QuerySet`` filters, where they make it +possible to filter a set of objects against criteria based on their field +values, rather than on Python values. + +This is documented in :ref:`using F() expressions in queries +` + +Supported operations with ``F()`` +--------------------------------- + +As well as addition, Django supports subtraction, multiplication, division, +and modulo arithmetic with ``F()`` objects, using Python constants, +variables, and even other ``F()`` objects. + +``Q()`` objects +=============== + +.. class:: Q + +A ``Q()`` object, like an :class:`~django.db.models.F` object, encapsulates a +SQL expression in a Python object that can be used in database-related +operations. + +In general, ``Q() objects`` make it possible to define and reuse conditions. +This permits the :ref:`construction of complex database queries +` using ``|`` (``OR``) and ``&`` (``AND``) operators; +in particular, it is not otherwise possible to use ``OR`` in ``QuerySets``. diff --git a/docs/releases/1.1-alpha-1.txt b/docs/releases/1.1-alpha-1.txt index b20b4103b8..e49b22c4aa 100644 --- a/docs/releases/1.1-alpha-1.txt +++ b/docs/releases/1.1-alpha-1.txt @@ -46,15 +46,15 @@ Query expressions Queries can now refer to a another field on the query and can traverse relationships to refer to fields on related models. This is implemented in the new :class:`F` object; for full details, including examples, consult the -:ref:`documentation for F expressions `. +:class:`F expressions documentation `. Performance improvements ------------------------ .. currentmodule:: django.test -Tests written using Django's :doc:`testing framework ` now run -dramatically faster (as much as 10 times faster in many cases). +Tests written using Django's :doc:`testing framework ` +now run dramatically faster (as much as 10 times faster in many cases). This was accomplished through the introduction of transaction-based tests: when using :class:`django.test.TestCase`, your tests will now be run in a transaction diff --git a/docs/releases/1.1.txt b/docs/releases/1.1.txt index ca8e1fff2a..261b552aff 100644 --- a/docs/releases/1.1.txt +++ b/docs/releases/1.1.txt @@ -211,7 +211,7 @@ Query expressions Queries can now refer to a another field on the query and can traverse relationships to refer to fields on related models. This is implemented in the new :class:`~django.db.models.F` object; for full details, including examples, -consult the :ref:`documentation for F expressions `. +consult the :class:`F expressions documentation `. Model improvements ------------------ diff --git a/docs/releases/1.3.txt b/docs/releases/1.3.txt index 6bf4c0db60..ecb48803a7 100644 --- a/docs/releases/1.3.txt +++ b/docs/releases/1.3.txt @@ -324,7 +324,7 @@ requests. These include: to :meth:`~django.shortcuts.render_to_response()` providing a :class:`~django.template.RequestContext` by default. -* Support for combining :ref:`F() expressions ` +* Support for combining :class:`F expressions ` with timedelta values when retrieving or updating database values. .. _HTTPOnly: https://www.owasp.org/index.php/HTTPOnly diff --git a/docs/releases/1.5-alpha-1.txt b/docs/releases/1.5-alpha-1.txt index 15169131df..df3df81d82 100644 --- a/docs/releases/1.5-alpha-1.txt +++ b/docs/releases/1.5-alpha-1.txt @@ -562,11 +562,12 @@ Miscellaneous needs. The new default value is ``0666`` (octal) and the current umask value is first masked out. -* The :ref:`F() expressions ` supported bitwise operators by - ``&`` and ``|``. These operators are now available using ``.bitand()`` and - ``.bitor()`` instead. The removal of ``&`` and ``|`` was done to be consistent with - :ref:`Q() expressions ` and ``QuerySet`` combining where - the operators are used as boolean AND and OR operators. +* The :class:`F expressions ` supported bitwise operators + by ``&`` and ``|``. These operators are now available using ``.bitand()`` and + ``.bitor()`` instead. The removal of ``&`` and ``|`` was done to be + consistent with :ref:`Q() expressions ` and + ``QuerySet`` combining where the operators are used as boolean AND and OR + operators. * The :ttag:`csrf_token` template tag is no longer enclosed in a div. If you need HTML validation against pre-HTML5 Strict DTDs, you should add a div around it diff --git a/docs/releases/1.5-beta-1.txt b/docs/releases/1.5-beta-1.txt index 4495a281af..8aa80ab6c1 100644 --- a/docs/releases/1.5-beta-1.txt +++ b/docs/releases/1.5-beta-1.txt @@ -601,13 +601,14 @@ Miscellaneous needs. The new default value is ``0666`` (octal) and the current umask value is first masked out. -* The :ref:`F() expressions ` supported bitwise operators by +* The :class:`F expressions ` supported bitwise operators by ``&`` and ``|``. These operators are now available using ``.bitand()`` and - ``.bitor()`` instead. The removal of ``&`` and ``|`` was done to be consistent with - :ref:`Q() expressions ` and ``QuerySet`` combining where - the operators are used as boolean AND and OR operators. + ``.bitor()`` instead. The removal of ``&`` and ``|`` was done to be + consistent with :ref:`Q() expressions ` and + ``QuerySet`` combining where the operators are used as boolean AND and OR + operators. -* In a ``filter()`` call, when :ref:`F() expressions ` +* In a ``filter()`` call, when :class:`F expressions ` contained lookups spanning multi-valued relations, they didn't always reuse the same relations as other lookups along the same chain. This was changed, and now F() expressions will always use the same relations as other lookups diff --git a/docs/releases/1.5.txt b/docs/releases/1.5.txt index 16d45de45f..bc5196f280 100644 --- a/docs/releases/1.5.txt +++ b/docs/releases/1.5.txt @@ -679,13 +679,14 @@ Miscellaneous needs. The new default value is ``0666`` (octal) and the current umask value is first masked out. -* The :ref:`F() expressions ` supported bitwise operators by +* The :class:`F expressions ` supported bitwise operators by ``&`` and ``|``. These operators are now available using ``.bitand()`` and - ``.bitor()`` instead. The removal of ``&`` and ``|`` was done to be consistent with - :ref:`Q() expressions ` and ``QuerySet`` combining where - the operators are used as boolean AND and OR operators. + ``.bitor()`` instead. The removal of ``&`` and ``|`` was done to be + consistent with :ref:`Q() expressions ` and + ``QuerySet`` combining where the operators are used as boolean AND and OR + operators. -* In a ``filter()`` call, when :ref:`F() expressions ` +* In a ``filter()`` call, when :class:`F expressions ` contained lookups spanning multi-valued relations, they didn't always reuse the same relations as other lookups along the same chain. This was changed, and now F() expressions will always use the same relations as other lookups diff --git a/docs/topics/db/optimization.txt b/docs/topics/db/optimization.txt index c1459ae247..c99c84d9fa 100644 --- a/docs/topics/db/optimization.txt +++ b/docs/topics/db/optimization.txt @@ -111,7 +111,7 @@ For instance: * At the most basic level, use :ref:`filter and exclude ` to do filtering in the database. -* Use :ref:`F() object query expressions ` to do filtering +* Use :class:`F expressions ` to do filtering against other fields within the same model. * Use :doc:`annotate to do aggregation in the database `. diff --git a/docs/topics/db/queries.txt b/docs/topics/db/queries.txt index 30335f266d..23f4ec7398 100644 --- a/docs/topics/db/queries.txt +++ b/docs/topics/db/queries.txt @@ -588,18 +588,16 @@ relation). Conditions in subsequent :meth:`~django.db.models.query.QuerySet.exclude` calls that refer to the same relation may end up filtering on different linked objects. -.. _query-expressions: +.. _using-f-expressions-in-filters: Filters can reference fields on the model ----------------------------------------- -.. class:: F - In the examples given so far, we have constructed filters that compare the value of a model field with a constant. But what if you want to compare the value of a model field with another field on the same model? -Django provides the :ref:`F() expressions ` to allow such +Django provides :class:`F expressions ` to allow such comparisons. Instances of ``F()`` act as a reference to a model field within a query. These references can then be used in query filters to compare the values of two different fields on the same model instance. @@ -779,15 +777,11 @@ being evaluated and therefore populate the cache:: Complex lookups with Q objects ============================== -.. class:: Q - Keyword argument queries -- in :meth:`~django.db.models.query.QuerySet.filter`, etc. -- are "AND"ed together. If you need to execute more complex queries (for -example, queries with ``OR`` statements), you can use ``Q`` objects. +example, queries with ``OR`` statements), you can use :class:`Q objects `. -.. comment: Link to Q does not work, since this documentation does not exist yet. - -A :class:`~django.db.models.Q` object (``django.db.models.Q``) is an object +A :class:`Q object ` (``django.db.models.Q``) is an object used to encapsulate a collection of keyword arguments. These keyword arguments are specified as in "Field lookups" above. @@ -1019,10 +1013,10 @@ over them and call :meth:`~django.db.models.Model.save`:: for item in my_queryset: item.save() -Calls to update can also use :ref:`F() objects ` to update -one field based on the value of another field in the model. This is especially -useful for incrementing counters based upon their current value. For example, to -increment the pingback count for every entry in the blog:: +Calls to update can also use :class:`F expressions ` to +update one field based on the value of another field in the model. This is +especially useful for incrementing counters based upon their current value. For +example, to increment the pingback count for every entry in the blog:: >>> Entry.objects.all().update(n_pingbacks=F('n_pingbacks') + 1)