The `remove()` and `clear()` methods of the related managers created by
`ForeignKey`, `GenericForeignKey`, and `ManyToManyField` suffered from a
number of issues. Some operations ran multiple data modifying queries without
wrapping them in a transaction, and some operations didn't respect default
filtering when it was present (i.e. when the default manager on the related
model implemented a custom `get_queryset()`).
Fixing the issues introduced some backward incompatible changes:
- The implementation of `remove()` for `ForeignKey` related managers changed
from a series of `Model.save()` calls to a single `QuerySet.update()` call.
The change means that `pre_save` and `post_save` signals aren't called anymore.
- The `remove()` and `clear()` methods for `GenericForeignKey` related
managers now perform bulk delete so `Model.delete()` isn't called anymore.
- The `remove()` and `clear()` methods for `ManyToManyField` related
managers perform nested queries when filtering is involved, which may
or may not be an issue depending on the database and the data itself.
Refs. #3871, #21174.
Thanks Anssi Kääriäinen and Tim Graham for the reviews.
This patch introduces the Prefetch object which allows customizing prefetch
operations.
This enables things like filtering prefetched relations, calling select_related
from a prefetched relation, or prefetching the same relation multiple times
with different querysets.
When a Prefetch instance specifies a to_attr argument, the result is stored
in a list rather than a QuerySet. This has the fortunate consequence of being
significantly faster. The preformance improvement is due to the fact that we
save the costly creation of a QuerySet instance.
Thanks @akaariai for the original patch and @bmispelon and @timgraham
for the reviews.
This commit introduced a new class JoinPromoter that can be used to
abstract away join promotion problems for complex filter conditions.
Query._add_q() and Query.combine() now use the new class.
Also, added a lot of comments about why join promotion is done the way
it is.
Thanks to Tim Graham for original report and testing the changes, and
for Loic Bistuer for review.
Thanks dan at dlo.me for the initial patch.
- Added __pow__ and __rpow__ to ExpressionNode
- Added oracle and mysql specific power expressions
- Added used-defined power function for sqlite
The typo could have consequences in exceptional cases, but I didn't
figure out a way to actually produce such a case, so not tests.
Report & patch by Michael Manfre.
select_related('foo').select_related('bar') is now equivalent to
select_related('foo', 'bar').
Also reworded docs to recommend select_related(*fields) over select_related()
Squashed commit of the following:
commit 63ddb271a44df389b2c302e421fc17b7f0529755
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 29 22:51:00 2013 +0200
Clarified interactions between atomic and exceptions.
commit 2899ec299228217c876ba3aa4024e523a41c8504
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:45:32 2013 +0200
Fixed TransactionManagementError in tests.
Previous commit introduced an additional check to prevent running
queries in transactions that will be rolled back, which triggered a few
failures in the tests. In practice using transaction.atomic instead of
the low-level savepoint APIs was enough to fix the problems.
commit 4a639b059ea80aeb78f7f160a7d4b9f609b9c238
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Sep 24 22:24:17 2013 +0200
Allowed nesting constraint_checks_disabled inside atomic.
Since MySQL handles transactions loosely, this isn't a problem.
commit 2a4ab1cb6e83391ff7e25d08479e230ca564bfef
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Sep 21 18:43:12 2013 +0200
Prevented running queries in transactions that will be rolled back.
This avoids a counter-intuitive behavior in an edge case on databases
with non-atomic transaction semantics.
It prevents using savepoint_rollback() inside an atomic block without
calling set_rollback(False) first, which is backwards-incompatible in
tests.
Refs #21134.
commit 8e3db393853c7ac64a445b66e57f3620a3fde7b0
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:14:17 2013 +0200
Replaced manual savepoints by atomic blocks.
This ensures the rollback flag is handled consistently in internal APIs.
Previously, if a database request spanned a related object manager, the
first manager encountered would cause a request to the router, and this
would bind all subsequent queries to the same database returned by the
router. Unfortunately, the first router query would be performed using
a read request to the router, resulting in bad routing information being
used if the subsequent query was actually a write.
This change defers the call to the router until the final query is acutally
made.
It includes a small *BACKWARDS INCOMPATIBILITY* on an edge case - see the
release notes for details.
Thanks to Paul Collins (@paulcollinsiii) for the excellent debugging
work and patch.
The use of OrderedDict (even an empty one) was surprisingly slow. By
initializing OrderedDict only when needed it is possible to save
non-trivial amount of computing time (Model.save() is around 30% faster
for example).
This commit targetted sql.Query only, there are likely other places
which could use similar optimizations.
The option can be used to force pre 1.6 style SELECT on save behaviour.
This is needed in case the database returns zero updated rows even if
there is a matching row in the DB. One such case is PostgreSQL update
trigger that returns NULL.
Reviewed by Tim Graham.
Refs #16649
The __eq__ method now considers two instances without primary key value
equal only when they have same id(). The __hash__ method raises
TypeError for no primary key case.
Fixed#18864, fixed#18250
Thanks to Tim Graham for docs review.
In cases where the same connection (from model A to model B along the
same field) was needed multiple times in a select_related query, the
join setup code mistakenly reused an existing join.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
The bug was already fixed by 01b9c3d519,
so only tests added.
At the same time promote_joins()'s uncoditional flag is gone, it isn't
needed for anything any more.
Loic Bistuer
Anssi Kääriäinen
Simon Charette
Baptiste Mispelon
Alex Gaynor
Florian Apolloner
Milton Mazzarri
coagulant
Tim Graham
Florian Hahn
Alasdair Nicol
Alasdair Nicol
Marc Tamlyn
Aymeric Augustin
Javed Khan
Russell Keith-Magee
Erik Romijn
Michael Manfre
Andrei Picus
Chris Wilson
Carl Meyer
Andrew Godwin
Alex Cucu
Curtis Maloney
Alex Couper