The Model.from_db() is intended to be used in cases where customization
of model loading is needed. Reasons can be performance, or adding custom
behavior to the model (for example "dirty field tracking" to issue
automatic update_fields when saving models).
A big thank you to Tim Graham for the review!
Regression from f51c1f59 when using select_related then prefetch_related
on the reverse side of an O2O:
Author.objects.select_related('bio').prefetch_related('bio__books')
Thanks Aymeric Augustin for the report and tests. Refs #17001.
Loading fixtures were failing since the refactoring in 244e2b71f5 for
inheritance setups where the chain contains abstract models and the
root ancestor contains a M2M relation.
Thanks Stanislas Guerra for the report.
Refs #20946.
Added DateTimeCheckMixin to avoid the use of default, auto_now, and
auto_now_add options together. Added the fields.E151 Error that is raised
if one or more of these options are used together.
Previously, known related objects overwrote related objects loaded
though select_related. This could cancel the effect of select_related
when it was used over more than one level.
Thanks boxm for the bug report and timo for bisecting the regression.
In some cases, this could lead to migrations written with Python 2
being incompatible with Python 3.
Thanks Tim Graham for the report and Loïc Bistuer for the advices.
Ordering by reverse foreign key was broken by custom lookups patch
(commit 20bab2cf9d).
Thanks to everybody who helped solving this issue. Special thanks to
Trac alias takis for reporting this.
When custom lookups were added, converting the search lookup to use
the new Lookup infrastructure wasn't done.
Some changes were needed to the added test, main change done by
committer was ensuring the test works on MySQL versions prior to 5.6.
So as the save step is centralized in create(), especially useful
when customizing behavior in subclasses.
Thanks craig.labenz@gmail.com for the report.
The ticket was originally about two failing tests, which are
fixed by putting their queries in transactions.
Thanks Tim Graham for the report, Aymeric Augustin for the fix,
and Simon Charette, Tim Graham & Loïc Bistuer for review.
Since assignments on M2M or reverse FK descriptors is composed of a `clear()`,
followed by an `add()`, `clear()` could potentially affect the value of the
assigned queryset before the `add()` step; pre-evaluating it solves the problem.
This patch fixes the issue for ForeignRelatedObjectsDescriptor,
ManyRelatedObjectsDescriptor, and ReverseGenericRelatedObjectsDescriptor.
It completes 6cb6e1 which addressed ReverseManyRelatedObjectsDescriptor.
db_parameters should respect an already existing db_type method and
return that as its type string. In particular, this was causing some
fields from gis to not be generated.
Thanks to @bigsassy and @blueyed for their work on the patch.
Also fixed#22260
Previously, saving a model instance to a non-related field (in
particular a FloatField) would silently convert the model to an Integer
(the pk) and save it. This is undesirable behaviour, and likely to cause
confusion so the validatio has been hardened.
Thanks to @PirosB3 for the patch and @jarshwah for the review.
GenericRelation now supports an optional related_query_name argument.
Setting related_query_name adds a relation from the related object back to
the content type for filtering, ordering and other query operations.
Thanks to Loic Bistuer for spotting a couple of important issues in
his review.
The original patch for custom prefetches didn't allow usage of custom
queryset for single valued relations (along ForeignKey or OneToOneKey).
Allowing these enables calling performance oriented queryset methods like
select_related or defer/only.
Thanks @akaariai and @timgraham for the reviews. Refs #17001.
Previously, doing so resulted in invalid data or crash.
Thanks jtiai for the report and Karol Jochelson,
Jakub Nowak, Loic Bistuer, and Baptiste Mispelon for reviews.
ForeignKey or ManyToManyField attribute ``limit_choices_to`` can now
be a callable that returns either a ``Q`` object or a dict.
Thanks michael at actrix.gen.nz for the original suggestion.
Overriding the error messages now works for both unique fields, unique_together
and unique_for_date.
This patch changed the overriding logic to allow customizing NON_FIELD_ERRORS
since previously only fields' errors were customizable.
Refs #20199.
Thanks leahculver for the suggestion.
This commit touchs various parts of the code base and test framework. Any
found usage of opening a cursor for the sake of initializing a connection
has been replaced with 'ensure_connection()'.
Updated SQLUpdateCompiler.execute_sql to match the behavior described in
the docstring; the 'first non-empty query' will now include all queries,
not just the main and first related update.
Added CURSOR and NO_RESULTS result_type constants to make the usages more
self documenting and allow execute_sql to explicitly close the cursor when
it is no longer needed.
The combination of BaseManager.from_queryset() and RenameMethodsBase results in
Manager.__module__ having the wrong value. This can be an issue when trying to
pickle the Manager class.
This is the result of Christopher Medrela's 2013 Summer of Code project.
Thanks also to Preston Holmes, Tim Graham, Anssi Kääriäinen, Florian
Apolloner, and Alex Gaynor for review notes along the way.
Also: Fixes#8579, fixes#3055, fixes#19844.
Allowed users to specify which lookups or transforms ("nested lookus")
are available for fields. The implementation is now class based.
Squashed commit of the following:
commit fa7a7195f1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 18 10:53:24 2014 +0200
Added lookup registration API docs
commit eb1c8ce164
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Tue Jan 14 18:59:36 2014 +0200
Release notes and other minor docs changes
commit 11501c29c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 20:53:03 2014 +0200
Forgot to add custom_lookups tests in prev commit
commit 83173b960e
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:59:12 2014 +0200
Renamed Extract -> Transform
commit 3b18d9f3a1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:51:53 2014 +0200
Removed suggestion of temporary lookup registration from docs
commit 21d0c7631c
Merge: 2509006f2dc442
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 09:38:23 2014 -0800
Merge pull request #2 from mjtamlyn/lookups_3
Reworked custom lookups docs.
commit f2dc4429a1
Author: Marc Tamlyn <marc.tamlyn@gmail.com>
Date: Sun Jan 12 13:15:05 2014 +0000
Reworked custom lookups docs.
Mostly just formatting and rewording, but also replaced the example
using ``YearExtract`` to use an example which is unlikely to ever be
possible directly in the ORM.
commit 2509006506
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 13:19:13 2014 +0200
Removed unused import
commit 4fba5dfaa0
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:34:41 2014 +0200
Added docs to index
commit 6d53963f37
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:10:24 2014 +0200
Dead code removal
commit f9cc039007
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 19:00:43 2014 +0200
A new try for docs
commit 33aa18a6e3
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:57:12 2014 +0200
Renamed get_cols to get_group_by_cols
commit c7d5f8661b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:45:53 2014 +0200
Altered query string customization for backends vendors
The new way is trying to call first method 'as_' + connection.vendor.
If that doesn't exist, then call as_sql().
Also altered how lookup registration is done. There is now
RegisterLookupMixin class that is used by Field, Extract and
sql.Aggregate. This allows one to register lookups for extracts and
aggregates in the same way lookup registration is done for fields.
commit 90e7004ec1
Merge: 66649fff7c2c0a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:21:01 2014 +0200
Merge branch 'master' into lookups_3
commit 66649ff891
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:16:01 2014 +0200
Some rewording in docs
commit 31b8faa627
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 29 15:52:29 2013 +0200
Cleanup based on review comments
commit 1016159f34
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 18:37:04 2013 +0200
Proof-of-concept fix for #16731
Implemented only for SQLite and PostgreSQL, and only for startswith
and istartswith lookups.
commit 193cd097ca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 17:57:58 2013 +0200
Fixed#11722 -- iexact=F() produced invalid SQL
commit 08ed3c3b49
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:59:52 2013 +0200
Made Lookup and Extract available from django.db.models
commit b99c8d83c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:06:29 2013 +0200
Fixed review notes by Loic
commit 049eebc070
Merge: ed8fab7b80a835
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:53:10 2013 +0200
Merge branch 'master' into lookups_3
Conflicts:
django/db/models/fields/__init__.py
django/db/models/sql/compiler.py
django/db/models/sql/query.py
tests/null_queries/tests.py
commit ed8fab7fe8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:47:23 2013 +0200
Made Extracts aware of full lookup path
commit 27a57b7aed
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:10:11 2013 +0200
Removed debugger import
commit 074e0f5aca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:02:16 2013 +0200
GIS lookup support added
commit 760e28e72b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 20:04:31 2013 +0200
Removed usage of Constraint, used Lookup instead
commit eac4776684
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:22:30 2013 +0200
Minor cleanup of Lookup API
commit 2adf50428d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:14:19 2013 +0200
Added documentation, polished implementation
commit 32c04357a8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:10:15 2013 +0200
Avoid OrderedDict creation on lookup aggregate check
commit 7c8b3a32cc
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:04:34 2013 +0200
Implemented nested lookups
But there is no support of using lookups outside filtering yet.
commit 4d219d4cde
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Wed Nov 27 22:07:30 2013 +0200
Initial implementation of custom lookups
Since it triggers imports, it shouldn't be done lightly.
This commit adds a public API for doing it explicitly, django.setup(),
and does it automatically when using manage.py and wsgi.py.
Returning None on errors required unpythonic error checking and was
inconsistent with get_app_config.
get_model was a private API until the previous commit, but given that it
was certainly used in third party software, the change is explained in
the release notes.
Applied the same change to get_registered_model, which is a new private
API introduced during the recent refactoring.
ContentTypes are only created for installed applications, and I could
make a case for not returning a model that isn't installed any more.
The check for stale ContentTypes in update_contenttypes doesn't use
model_class.
ModelSignal actually needs get_registered_model since the lookup happens
at import time. I took this opportunity to perform a small refactoring.
It was called _populate() before I renamed it to populate(). Since it
has been superseded by populate_models() there's no reason to keep it.
Removed the can_postpone argument of load_app() as it was only used by
populate(). It's a private API and there's no replacement. Simplified
load_app() accordingly. Then new version behaves exactly like the old
one even though it's much shorter.
Since applications that aren't installed no longer have an application
configuration, it is now always True in practice.
Provided an abstraction to temporarily add or remove applications as
several tests messed with app_config.installed to achieve this effect.
For now this API is _-prefixed because it looks dangerous.
Got rid of AppConfig._stub. As a side effect, app_cache.app_configs now
only contains entries for applications that are in INSTALLED_APPS, which
is a good thing and will allow dramatic simplifications (which I will
perform in the next commit). That required adjusting all methods that
iterate on app_configs without checking the "installed" flag, hence the
large changes in get_model[s].
Introduced AppCache.all_models to store models:
- while the app cache is being populated and a suitable app config
object to register models isn't available yet;
- for applications that aren't in INSTALLED_APPS since they don't have
an app config any longer.
Replaced get_model(seed_cache=False) by registered_model() which can be
kept simple and safe to call at any time, and removed the seed_cache
argument to get_model[s]. There's no replacement for that private API.
Allowed non-master app caches to go through populate() as it is now
safe to do so. They were introduced in 1.7 so backwards compatibility
isn't a concern as long as the migrations framework keeps working.
Used the information from the app cache instead of creating a duplicate
based on INSTALLED_APPS.
Model._meta.installed is no longer writable. It was a rather sketchy way
to alter private internals anyway.
Several parts of Django call get_apps() with a comment along this lines
of "this has the side effect of calling _populate()". I fail to see how
this is better than just calling populate()!
Since the original ones in django.db.models.loading were kept only for
backwards compatibility, there's no need to recreate them. However, many
internals of Django still relied on them.
They were also imported in django.db.models. They never appear in the
documentation, except a quick mention of get_models and get_app in the
1.2 release notes to document an edge case in GIS. I don't think that
makes them a public API.
This commit doesn't change the overall amount of global state but
clarifies that it's tied to the app_cache object instead of hiding it
behind half a dozen functions.
The `remove()` and `clear()` methods of the related managers created by
`ForeignKey`, `GenericForeignKey`, and `ManyToManyField` suffered from a
number of issues. Some operations ran multiple data modifying queries without
wrapping them in a transaction, and some operations didn't respect default
filtering when it was present (i.e. when the default manager on the related
model implemented a custom `get_queryset()`).
Fixing the issues introduced some backward incompatible changes:
- The implementation of `remove()` for `ForeignKey` related managers changed
from a series of `Model.save()` calls to a single `QuerySet.update()` call.
The change means that `pre_save` and `post_save` signals aren't called anymore.
- The `remove()` and `clear()` methods for `GenericForeignKey` related
managers now perform bulk delete so `Model.delete()` isn't called anymore.
- The `remove()` and `clear()` methods for `ManyToManyField` related
managers perform nested queries when filtering is involved, which may
or may not be an issue depending on the database and the data itself.
Refs. #3871, #21174.
Thanks Anssi Kääriäinen and Tim Graham for the reviews.
This patch introduces the Prefetch object which allows customizing prefetch
operations.
This enables things like filtering prefetched relations, calling select_related
from a prefetched relation, or prefetching the same relation multiple times
with different querysets.
When a Prefetch instance specifies a to_attr argument, the result is stored
in a list rather than a QuerySet. This has the fortunate consequence of being
significantly faster. The preformance improvement is due to the fact that we
save the costly creation of a QuerySet instance.
Thanks @akaariai for the original patch and @bmispelon and @timgraham
for the reviews.
This commit introduced a new class JoinPromoter that can be used to
abstract away join promotion problems for complex filter conditions.
Query._add_q() and Query.combine() now use the new class.
Also, added a lot of comments about why join promotion is done the way
it is.
Thanks to Tim Graham for original report and testing the changes, and
for Loic Bistuer for review.
Thanks dan at dlo.me for the initial patch.
- Added __pow__ and __rpow__ to ExpressionNode
- Added oracle and mysql specific power expressions
- Added used-defined power function for sqlite
The typo could have consequences in exceptional cases, but I didn't
figure out a way to actually produce such a case, so not tests.
Report & patch by Michael Manfre.
select_related('foo').select_related('bar') is now equivalent to
select_related('foo', 'bar').
Also reworded docs to recommend select_related(*fields) over select_related()
Squashed commit of the following:
commit 63ddb271a44df389b2c302e421fc17b7f0529755
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 29 22:51:00 2013 +0200
Clarified interactions between atomic and exceptions.
commit 2899ec299228217c876ba3aa4024e523a41c8504
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:45:32 2013 +0200
Fixed TransactionManagementError in tests.
Previous commit introduced an additional check to prevent running
queries in transactions that will be rolled back, which triggered a few
failures in the tests. In practice using transaction.atomic instead of
the low-level savepoint APIs was enough to fix the problems.
commit 4a639b059ea80aeb78f7f160a7d4b9f609b9c238
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Sep 24 22:24:17 2013 +0200
Allowed nesting constraint_checks_disabled inside atomic.
Since MySQL handles transactions loosely, this isn't a problem.
commit 2a4ab1cb6e83391ff7e25d08479e230ca564bfef
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Sep 21 18:43:12 2013 +0200
Prevented running queries in transactions that will be rolled back.
This avoids a counter-intuitive behavior in an edge case on databases
with non-atomic transaction semantics.
It prevents using savepoint_rollback() inside an atomic block without
calling set_rollback(False) first, which is backwards-incompatible in
tests.
Refs #21134.
commit 8e3db393853c7ac64a445b66e57f3620a3fde7b0
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:14:17 2013 +0200
Replaced manual savepoints by atomic blocks.
This ensures the rollback flag is handled consistently in internal APIs.
Previously, if a database request spanned a related object manager, the
first manager encountered would cause a request to the router, and this
would bind all subsequent queries to the same database returned by the
router. Unfortunately, the first router query would be performed using
a read request to the router, resulting in bad routing information being
used if the subsequent query was actually a write.
This change defers the call to the router until the final query is acutally
made.
It includes a small *BACKWARDS INCOMPATIBILITY* on an edge case - see the
release notes for details.
Thanks to Paul Collins (@paulcollinsiii) for the excellent debugging
work and patch.
The use of OrderedDict (even an empty one) was surprisingly slow. By
initializing OrderedDict only when needed it is possible to save
non-trivial amount of computing time (Model.save() is around 30% faster
for example).
This commit targetted sql.Query only, there are likely other places
which could use similar optimizations.
The option can be used to force pre 1.6 style SELECT on save behaviour.
This is needed in case the database returns zero updated rows even if
there is a matching row in the DB. One such case is PostgreSQL update
trigger that returns NULL.
Reviewed by Tim Graham.
Refs #16649
The __eq__ method now considers two instances without primary key value
equal only when they have same id(). The __hash__ method raises
TypeError for no primary key case.
Fixed#18864, fixed#18250
Thanks to Tim Graham for docs review.
In cases where the same connection (from model A to model B along the
same field) was needed multiple times in a select_related query, the
join setup code mistakenly reused an existing join.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
The bug was already fixed by 01b9c3d519,
so only tests added.
At the same time promote_joins()'s uncoditional flag is gone, it isn't
needed for anything any more.
All Promise objects were passed to force_text() deep in ORM query code.
Not only does this make it difficult or impossible for developers to
prevent or alter this behaviour, but it is also wrong for non-text
fields.
This commit changes `Field.get_prep_value()` from a no-op to one that
resolved Promise objects. All subclasses now call super() method first
to ensure that they have a real value to work with.
It has been possible to use models of wrong type in related field
lookups. For example pigs__in=[a_duck] has worked. Changes to
ForeignObject broke that.
It might be a good idea to restrict the model types usable in lookups.
This should be done intentionally, not accidentally and without any
consideration for deprecation path.
Additionally this patch solves the orthogonal problem that specialized
`QuerySet` like `ValuesQuerySet` didn't inherit from the current `QuerySet`
type. This wasn't an issue until now because we didn't officially support
custom `QuerySet` but it became necessary with the introduction of this new
feature.
Thanks aaugustin, akaariai, carljm, charettes, mjtamlyn, shaib and timgraham
for the reviews.
In the combination of .values().aggregate() the aggregate_select_mask
didn't include the aggregates added. This resulted in bogus query.
Thanks to Trac alias debanshuk for report.
In the intervening years, RelatedField has become less of a hack (though it still is one). Anyone who wants to can re-instate the comment, but please add more details.
Add support for Oracle, fix an issue with the repr of RawQuerySet,
add tests and documentations. Also added a 'supports_paramstyle_pyformat'
database feature, True by default, False for SQLite.
Thanks Donald Stufft for review of documentation.
Those methods were only used by `contrib.admin` internally and exclusively
related to `contrib.auth`. Since they were undocumented but used
in the wild the raised deprecation warning point to an also undocumented
alternative that lives in `contrib.auth`.
Also did some PEP8 and other cleanups in the affected modules.
There were a couple of places which used Query.join() directly. By
using setup_joins() in these places the code is more DRY, and in
addition there is no need to directly call field.get_joining_columns()
unless the field is the given join_field from get_path_info(). This
makes it easier to make sure a ForeignObject subclass generates joins
correctly in all cases.
The join used by select_related was incorrectly INNER when the query
had an ORed filter for nullable join that was trimmed away. Fixed this
by forcing the join type to LOUTER even when a join was trimmed away
in ORed queries.
This can be used to make Django's test suite significantly faster by
reducing the number of models for which content types and permissions
must be created and tables must be flushed in each non-transactional
test.
It's documented for Django contributors and committers but it's branded
as a private API to preserve our freedom to change it in the future.
Most of the credit goes to Anssi. He got the idea and did the research.
Fixed#20483.
The patch for #19385 caused a regression in certain generic relations
.exclude() filters if a subquery was needed. The fix contains a
refactoring to how Query.split_exclude() and Query.trim_start()
interact.
Thanks to Trac alias nferrari for the report.
Correctly calculate the ``aggregate_start`` offset from loaded fields,
if any are deferred, instead of ``self.query.select`` which includes all
fields on the model.
Also made some PEP 8 fixes.
The SubqueryConstraint defined relabeled_clone(), but that was never
called. Instead there is now clone() and relabel_aliases() methods for
SubqueryConstraint.
A related problem was that SubqueryConstraint didn't correctly use
quote_name_unless_alias() of the outer query. This resulted in failures
when running under PostgreSQL.
When an exception other than IntegrityError was raised, get_or_create
could fail and leave the database connection in an unusable state.
Thanks UloPe for the report.
This is backward incompatible for custom form field/widgets that rely
on the hard-coded 'Hold down "Control", or "Command" on a Mac, to select
more than one.' sentence.
Application that use standard model form fields and widgets aren't
affected but need to start handling these help texts by themselves
before Django 1.8.
For more details, see the related release notes and deprecation timeline
sections added with this commit.
A regression caused by d5b93d3281 made .get() error
reporting recurse infinitely on certain rare conditions. Fixed this by
not trying to print the given lookup kwargs.
When a GenericRelation was defined on abstract model, queries on childs
of the abstract model didn't work. The problem was in the way fields and
in particular field.rel was copied from models to their children.
The regression was likely caused by #19385. Thanks to Gavin Wahl for
spotting the regression.
This reverts commit 2cd0edaa47.
This commit was the cause of a memory leak. See ticket for more details.
Thanks Anssi Kääriäinen for identifying the source of the bug.
Model.save() will use UPDATE - if not updated - INSERT instead of
SELECT - if found UPDATE else INSERT. This should save a query when
updating, but will cost a little when inserting model with PK set.
Also fixed#17341 -- made sure .save() commits transactions only after
the whole model has been saved. This wasn't the case in model
inheritance situations.
The save_base implementation was refactored into multiple methods.
A typical chain for inherited save is:
save_base()
_save_parents(self)
for each parent:
_save_parents(parent)
_save_table(parent)
_save_table(self)
The sql/query.py add_q method did a lot of where/having tree hacking to
get complex queries to work correctly. The logic was refactored so that
it should be simpler to understand. The new logic should also produce
leaner WHERE conditions.
The changes cascade somewhat, as some other parts of Django (like
add_filter() and WhereNode) expect boolean trees in certain format or
they fail to work. So to fix the add_q() one must fix utils/tree.py,
some things in add_filter(), WhereNode and so on.
This commit also fixed add_filter to see negate clauses up the path.
A query like .exclude(Q(reversefk__in=a_list)) didn't work similarly to
.filter(~Q(reversefk__in=a_list)). The reason for this is that only
the immediate parent negate clauses were seen by add_filter, and thus a
tree like AND: (NOT AND: (AND: condition)) will not be handled
correctly, as there is one intermediary AND node in the tree. The
example tree is generated by .exclude(~Q(reversefk__in=a_list)).
Still, aggregation lost connectors in OR cases, and F() objects and
aggregates in same filter clause caused GROUP BY problems on some
databases.
Fixed#17600, fixed#13198, fixed#17025, fixed#17000, fixed#11293.
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
Since "unless managed" now means "if database-level autocommit",
committing or rolling back doesn't have any effect.
Restored transactional integrity in a few places that relied on
automatically-started transactions with a transitory API.
enter_transaction_management() was nearly always followed by managed().
In three places it wasn't, but they will all be refactored eventually.
The "forced" keyword argument avoids introducing behavior changes until
then.
This is mostly backwards-compatible, except, of course, for managed
itself. There's a minor difference in _enter_transaction_management:
the top self.transaction_state now contains the new 'managed' state
rather than the previous one. Django doesn't access
self.transaction_state in _enter_transaction_management.
There were a couple of errors in ._dirty flag handling:
* It started as None, but was never reset to None.
* The _dirty flag was sometimes used to indicate if the connection
was inside transaction management, but this was not done
consistently. This also meant the flag had three separate values.
* The None value had a special meaning, causing for example inability
to commit() on new connection unless enter/leave tx management was
done.
* The _dirty was tracking "connection in transaction" state, but only
in managed transactions.
* Some tests never reset the transaction state of the used connection.
* And some additional less important changes.
This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.
When iteration over a queryset raised an exception, the result cache
remained initialized with an empty list, so subsequent iterations returned
an empty list instead of raising an exception
Querying the reverse side of nullable to_field relation, where both
sides can contain null values resulted in incorrect results. The reason
was not detecting '' as NULL.
Refs #17541
There was a regression in case two models inherited the same parent,
and one contained a foreign key to other. When select_related travelled
the foreign key the other model reused the parent join made by the
first model. This was likely caused by Query.join_parent_model()
addition in commit 68985db482.
Thanks to Trac alias loic84 for report & tests.
The join promote=True was over-aggressive in select_related handling.
After that was removed, the only other user was query.combine(). That
use case is very easy to handle locally, so there is no more need for
the join(promote=True) flag.
Refs #19849.
This controls whether or not a database level cosntraint is created. This is useful in a few specialized circumstances, but in general should not be used!
The refactoring mainly concentrates on making sure the inner and outer
query agree about the split position. The split position is where the
multijoin happens, and thus the split position also determines the
columns used in the "WHERE col1 IN (SELECT col2 from ...)" condition.
This commit fixes a regression caused by #10790 and commit
69597e5bcc. The regression was caused
by wrong cols in the split position.
Thanks Carl Meyer for the review.
Squashed commit of the following:
commit 4f290bdb60
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 21:21:30 2013 +0100
Used '0:00' instead of 'UTC' which doesn't always exist in Oracle.
Thanks Ian Kelly for the suggestion.
commit 01b6366f3c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 13:38:43 2013 +0100
Made tzname a parameter of datetime_extract/trunc_sql.
This is required to work around a bug in Oracle.
commit 924a144ef8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 14:47:44 2013 +0100
Added support for parameters in SELECT clauses.
commit b4351d2890
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 22:30:22 2013 +0100
Documented backwards incompatibilities in the two previous commits.
commit 91ef84713c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:42:31 2013 +0100
Used QuerySet.datetimes for the admin's date_hierarchy.
commit 0d0de288a5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:29:38 2013 +0100
Used QuerySet.datetimes in date-based generic views.
commit 9c0859ff7c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:25 2013 +0100
Implemented QuerySet.datetimes on Oracle.
commit 68ab511a4f
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:14 2013 +0100
Implemented QuerySet.datetimes on MySQL.
commit 22d52681d3
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:42:29 2013 +0100
Implemented QuerySet.datetimes on SQLite.
commit f6800fd04c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:03 2013 +0100
Implemented QuerySet.datetimes on PostgreSQL.
commit 0c829c23f4
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:41:08 2013 +0100
Added datetime-handling infrastructure in the ORM layers.
commit 104d82a777
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 10:05:55 2013 +0100
Updated null_queries tests to avoid clashing with the __second lookup.
commit c01bbb3235
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 23:07:41 2013 +0100
Updated tests of .dates().
Replaced .dates() by .datetimes() for DateTimeFields.
Replaced dates with datetimes in the expected output for DateFields.
commit 50fb7a5246
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:40:09 2013 +0100
Updated and added tests for QuerySet.datetimes.
commit a8451a5004
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 22:34:46 2013 +0100
Documented the new time lookups and updated the date lookups.
commit 29413eab2b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 16:15:49 2013 +0100
Documented QuerySet.datetimes and updated QuerySet.dates.
value_annotation isn't very well defined. Before this change, setting it
to datetime.datetime could silently reverse the behavior of isnull
lookups. This commit doesn't have any consequences on the current code.
It's just a safeguard for future ORM hackers.
Previously it was possible to call clear_ordering without the
force_empty argument. The result was that the query was still ordered
by model's meta ordering if that was defined. By making the arg
mandatory it will be easier to spot possible errors caused by assuming
clear_ordering will remove all ordering.
Thanks to Dylan Klomparens for the suggestion. Refs #19720.
When a query had a complex where condition (a condition targeting more
than the base table) a subquery was used for deletion. However, the
query had default ordering from the model's meta and Oracle doesn't
work with ordered subqueries.
The regression was caused by fast-path deletion code introduced in
1cd6e04cd4 for fixing #18676.
Thanks to Dylan Klomparens for the report.
This undocumented method was used in an old version of the admin, is
totally untested and hails from 2008. Although it's listed in the
"public methods" section, as it's not documented or used I don't think
it needs a deprecation path.
If we think it's useful I'll write some tests/docs for it instead...
Thanks a lot to everybody participating in developing this feature.
The patch was developed by multiple people, at least Trac aliases
tonnzor, jimmysong, Fandekasp and slurms.
Stylistic changes added by committer.
The class wasn't used anywhere except in RelatedObject.bind(), which
wasn't used anywhere. The class had one method defined as
NotImplemented, yet the class wasn't subclassed anywhere. In short, the
class was dead code.
In a normal relational construct, if you're listening for an event
that signals a child was deleted, you dont expect that the parent
was deleted already.
This change ensures that post_delete signals are fired immediately
after objects are deleted in the graph.
The original problem was that queryset cloning was really expensive
when filtering with F() clauses. The __deepcopy__ went too deep copying
_meta attributes of the models used. To fix this the use of
__deepcopy__ in qs cloning was removed.
This commit results in some speed improvements across the djangobench
benchmark suite. Most query_* tests are 20-30% faster, save() is 50%
faster and finally complex filtering situations can see 2x to order
of magnitude improvments.
Thanks to Suor, Alex and lrekucki for valuable feedback.
The guarantee that no queries will be made when accessing results is
done by new EmptyWhere class which is used for query.where and having.
Thanks to Simon Charette for reviewing and valuable suggestions.
When &'ing or |'ing querysets, wrong values could be cached, and crashes
could happen.
Thanks Marc Tamlyn for figuring out the problem and writing the patch.
The added promotion logic is based on promoting any joins used in only
some of the childs of an OR clause unless the join existed before the
OR clause addition.
The ORM didn't reuse joins for direct foreign key traversals when using
chained filters. For example:
qs.filter(fk__somefield=1).filter(fk__somefield=2))
produced two joins.
As a bonus, reverse onetoone filters can now reuse joins correctly
The regression was caused by the join() method refactor in commit
68847135bc
Thanks for Simon Charette for spotting some issues with the first draft
of the patch.
This is necessary because get_model() checks are case insensitive, and if the swapable check isn't, the
swappable logic gets tied up in knots with models that are partially swapped out.
Thanks to chris@cogdon.org for the report and extensive analysis, and Preston for his work on the draft patch.
This is a rather large refactoring. The "lookup traversal" code was
splitted out from the setup_joins. There is now names_to_path() method
which does the lookup traveling, the actual work of setup_joins() is
calling names_to_path() and then adding the joins found into the query.
As a side effect it was possible to remove the "process_extra"
functionality used by genric relations. This never worked for left
joins. Now the extra restriction is appended directly to the join
condition instead of the where clause.
To generate the extra condition we need to have the join field
available in the compiler. This has the side-effect that we need more
ugly code in Query.__getstate__ and __setstate__ as Field objects
aren't pickleable.
The join trimming code got a big change - now we trim all direct joins
and never trim reverse joins. This also fixes the problem in #10790
which was join trimming in null filter cases.
The trim argument was used by split_exclude() only to trim the last
join from the given lookup. It is cleaner to just trim the last part
from the lookup in split_exclude() directly so that there is no need
to burden add_filter() with the logic needed for only split_exclude().
Anubhav Joshi
Daniel Hahler
Tim Graham
Anssi Kääriäinen
Lovas Bence
Andrew Godwin
Renaud Parent
Jorge C. Leitão
Craig de Stigter
Vincent-Vega
Alex Gaynor
Víðir Valberg Guðmundsson
Ramiro Morales
Loic Bistuer
Ramiro Morales
Marc Tamlyn
Florian Apolloner
Markus Holtermann
Vlad Starostin
Russell Keith-Magee
Daniel Pyrathon
Aymeric Augustin
Claude Paroz
Jakub Roztocil
Yehonatan Daniv
Simon Charette
Erik Romijn
Justin Hamade
Marc Egli
valtron
Shai Berger
Baptiste Mispelon
Akis Kesoglou
Gabe Jackson
Akis Kesoglou
Chris Wilson
Loic Bistuer
Rodolfo Carvalho
Martin Matusiak
Artur Frysiak
Christopher Medrela
Albert Wang
Christopher Adams
Robert Stapenhurst
Michael Manfre
maurycyp
Jonas Gastal
Alexey Voronov
Denis Moskalets
Roger Hu
pegler
Alex Hill
Milton Mazzarri
coagulant
Florian Hahn
Alasdair Nicol
Alasdair Nicol
Javed Khan
Erik Romijn
Andrei Picus
Carl Meyer
Alex Cucu
Curtis Maloney
Alex Couper
Tai Lee
Julien Phalip
ersran9
Karol Sikora
Dan Loewenherz
Tim Graham
Luke Plant
Donald Stufft
Selwin Ong
Mike Fogel
Mike Fogel
Adrian Holovaty
Tobias McNulty
Alex Gaynor
Gavin Wahl
Daniele Procida
Honza Kral
Marcin Biernat
Grzegorz Nosek
Tomek Paczkowski
Nick Sandford
David Cramer
mpaolini
orblivion