This is the result of Christopher Medrela's 2013 Summer of Code project.
Thanks also to Preston Holmes, Tim Graham, Anssi Kääriäinen, Florian
Apolloner, and Alex Gaynor for review notes along the way.
Also: Fixes#8579, fixes#3055, fixes#19844.
Allowed users to specify which lookups or transforms ("nested lookus")
are available for fields. The implementation is now class based.
Squashed commit of the following:
commit fa7a7195f1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 18 10:53:24 2014 +0200
Added lookup registration API docs
commit eb1c8ce164
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Tue Jan 14 18:59:36 2014 +0200
Release notes and other minor docs changes
commit 11501c29c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 20:53:03 2014 +0200
Forgot to add custom_lookups tests in prev commit
commit 83173b960e
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:59:12 2014 +0200
Renamed Extract -> Transform
commit 3b18d9f3a1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:51:53 2014 +0200
Removed suggestion of temporary lookup registration from docs
commit 21d0c7631c
Merge: 2509006f2dc442
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 09:38:23 2014 -0800
Merge pull request #2 from mjtamlyn/lookups_3
Reworked custom lookups docs.
commit f2dc4429a1
Author: Marc Tamlyn <marc.tamlyn@gmail.com>
Date: Sun Jan 12 13:15:05 2014 +0000
Reworked custom lookups docs.
Mostly just formatting and rewording, but also replaced the example
using ``YearExtract`` to use an example which is unlikely to ever be
possible directly in the ORM.
commit 2509006506
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 13:19:13 2014 +0200
Removed unused import
commit 4fba5dfaa0
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:34:41 2014 +0200
Added docs to index
commit 6d53963f37
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:10:24 2014 +0200
Dead code removal
commit f9cc039007
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 19:00:43 2014 +0200
A new try for docs
commit 33aa18a6e3
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:57:12 2014 +0200
Renamed get_cols to get_group_by_cols
commit c7d5f8661b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:45:53 2014 +0200
Altered query string customization for backends vendors
The new way is trying to call first method 'as_' + connection.vendor.
If that doesn't exist, then call as_sql().
Also altered how lookup registration is done. There is now
RegisterLookupMixin class that is used by Field, Extract and
sql.Aggregate. This allows one to register lookups for extracts and
aggregates in the same way lookup registration is done for fields.
commit 90e7004ec1
Merge: 66649fff7c2c0a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:21:01 2014 +0200
Merge branch 'master' into lookups_3
commit 66649ff891
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:16:01 2014 +0200
Some rewording in docs
commit 31b8faa627
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 29 15:52:29 2013 +0200
Cleanup based on review comments
commit 1016159f34
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 18:37:04 2013 +0200
Proof-of-concept fix for #16731
Implemented only for SQLite and PostgreSQL, and only for startswith
and istartswith lookups.
commit 193cd097ca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 17:57:58 2013 +0200
Fixed#11722 -- iexact=F() produced invalid SQL
commit 08ed3c3b49
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:59:52 2013 +0200
Made Lookup and Extract available from django.db.models
commit b99c8d83c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:06:29 2013 +0200
Fixed review notes by Loic
commit 049eebc070
Merge: ed8fab7b80a835
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:53:10 2013 +0200
Merge branch 'master' into lookups_3
Conflicts:
django/db/models/fields/__init__.py
django/db/models/sql/compiler.py
django/db/models/sql/query.py
tests/null_queries/tests.py
commit ed8fab7fe8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:47:23 2013 +0200
Made Extracts aware of full lookup path
commit 27a57b7aed
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:10:11 2013 +0200
Removed debugger import
commit 074e0f5aca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:02:16 2013 +0200
GIS lookup support added
commit 760e28e72b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 20:04:31 2013 +0200
Removed usage of Constraint, used Lookup instead
commit eac4776684
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:22:30 2013 +0200
Minor cleanup of Lookup API
commit 2adf50428d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:14:19 2013 +0200
Added documentation, polished implementation
commit 32c04357a8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:10:15 2013 +0200
Avoid OrderedDict creation on lookup aggregate check
commit 7c8b3a32cc
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:04:34 2013 +0200
Implemented nested lookups
But there is no support of using lookups outside filtering yet.
commit 4d219d4cde
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Wed Nov 27 22:07:30 2013 +0200
Initial implementation of custom lookups
Returning None on errors required unpythonic error checking and was
inconsistent with get_app_config.
get_model was a private API until the previous commit, but given that it
was certainly used in third party software, the change is explained in
the release notes.
Applied the same change to get_registered_model, which is a new private
API introduced during the recent refactoring.
Got rid of AppConfig._stub. As a side effect, app_cache.app_configs now
only contains entries for applications that are in INSTALLED_APPS, which
is a good thing and will allow dramatic simplifications (which I will
perform in the next commit). That required adjusting all methods that
iterate on app_configs without checking the "installed" flag, hence the
large changes in get_model[s].
Introduced AppCache.all_models to store models:
- while the app cache is being populated and a suitable app config
object to register models isn't available yet;
- for applications that aren't in INSTALLED_APPS since they don't have
an app config any longer.
Replaced get_model(seed_cache=False) by registered_model() which can be
kept simple and safe to call at any time, and removed the seed_cache
argument to get_model[s]. There's no replacement for that private API.
Allowed non-master app caches to go through populate() as it is now
safe to do so. They were introduced in 1.7 so backwards compatibility
isn't a concern as long as the migrations framework keeps working.
The `remove()` and `clear()` methods of the related managers created by
`ForeignKey`, `GenericForeignKey`, and `ManyToManyField` suffered from a
number of issues. Some operations ran multiple data modifying queries without
wrapping them in a transaction, and some operations didn't respect default
filtering when it was present (i.e. when the default manager on the related
model implemented a custom `get_queryset()`).
Fixing the issues introduced some backward incompatible changes:
- The implementation of `remove()` for `ForeignKey` related managers changed
from a series of `Model.save()` calls to a single `QuerySet.update()` call.
The change means that `pre_save` and `post_save` signals aren't called anymore.
- The `remove()` and `clear()` methods for `GenericForeignKey` related
managers now perform bulk delete so `Model.delete()` isn't called anymore.
- The `remove()` and `clear()` methods for `ManyToManyField` related
managers perform nested queries when filtering is involved, which may
or may not be an issue depending on the database and the data itself.
Refs. #3871, #21174.
Thanks Anssi Kääriäinen and Tim Graham for the reviews.
This patch introduces the Prefetch object which allows customizing prefetch
operations.
This enables things like filtering prefetched relations, calling select_related
from a prefetched relation, or prefetching the same relation multiple times
with different querysets.
When a Prefetch instance specifies a to_attr argument, the result is stored
in a list rather than a QuerySet. This has the fortunate consequence of being
significantly faster. The preformance improvement is due to the fact that we
save the costly creation of a QuerySet instance.
Thanks @akaariai for the original patch and @bmispelon and @timgraham
for the reviews.
Previously, if a database request spanned a related object manager, the
first manager encountered would cause a request to the router, and this
would bind all subsequent queries to the same database returned by the
router. Unfortunately, the first router query would be performed using
a read request to the router, resulting in bad routing information being
used if the subsequent query was actually a write.
This change defers the call to the router until the final query is acutally
made.
It includes a small *BACKWARDS INCOMPATIBILITY* on an edge case - see the
release notes for details.
Thanks to Paul Collins (@paulcollinsiii) for the excellent debugging
work and patch.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
It has been possible to use models of wrong type in related field
lookups. For example pigs__in=[a_duck] has worked. Changes to
ForeignObject broke that.
It might be a good idea to restrict the model types usable in lookups.
This should be done intentionally, not accidentally and without any
consideration for deprecation path.
In the intervening years, RelatedField has become less of a hack (though it still is one). Anyone who wants to can re-instate the comment, but please add more details.
This is backward incompatible for custom form field/widgets that rely
on the hard-coded 'Hold down "Control", or "Command" on a Mac, to select
more than one.' sentence.
Application that use standard model form fields and widgets aren't
affected but need to start handling these help texts by themselves
before Django 1.8.
For more details, see the related release notes and deprecation timeline
sections added with this commit.
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
Querying the reverse side of nullable to_field relation, where both
sides can contain null values resulted in incorrect results. The reason
was not detecting '' as NULL.
Refs #17541
This controls whether or not a database level cosntraint is created. This is useful in a few specialized circumstances, but in general should not be used!
When &'ing or |'ing querysets, wrong values could be cached, and crashes
could happen.
Thanks Marc Tamlyn for figuring out the problem and writing the patch.
The dupe avoidance logic was removed as it doesn't seem to do anything,
it is complicated, and it has nearly zero documentation.
The removal of dupe_avoidance allowed for refactoring of both the
implementation and signature of Query.join(). This refactoring cascades
again to some other parts. The most significant of them is the changes
in qs.combine(), and compiler.select_related_descent().
There was a potential data-loss issue involved -- when clearing
instance's m2m assignments it was possible some other instance's
m2m data was deleted instead.
This commit also improved None handling for to_field cases.
Thanks to the many people that contributed to the development and review of
this patch, including (but not limited to) Jacob Kaplan-Moss, Anssi
Kääriäinen, Ramiro Morales, Preston Holmes, Josh Ourisman, Thomas Sutton,
and Roger Barnes, as well as the many, many people who have contributed to
the design discussion around this ticket over many years.
Squashed commit of the following:
commit d84749a0f0
Merge: 531e7717c11b1a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Wed Sep 26 18:37:04 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 531e7715da
Merge: 29d1abb1f84b04
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Wed Sep 26 07:09:23 2012 +0800
Merged recent trunk changes.
commit 29d1abbe35
Merge: 8a527dd54c81a1
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 24 07:49:46 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 8a527dda13
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 24 07:48:05 2012 +0800
Ensure sequences are reset correctly in the presence of swapped models.
commit e2b6e22f29
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 17:53:05 2012 +0800
Modifications to the handling and docs for auth forms.
commit 98aba856b5
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 15:28:57 2012 +0800
Improved error handling and docs for get_user_model()
commit 0229209c84
Merge: 6494bf98599f64
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 14:50:11 2012 +0800
Merged recent Django trunk changes.
commit 6494bf91f2
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 17 21:38:44 2012 +0800
Improved validation of swappable model settings.
commit 5a04cde342
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 17 07:15:14 2012 +0800
Removed some unused imports.
commit ffd535e413
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 20:31:28 2012 +0800
Corrected attribute access on for get_by_natural_key
commit 913e1ac84c
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 20:12:34 2012 +0800
Added test for proxy model safeguards on swappable models.
commit 280bf19e94
Merge: dbb3900935a863
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 18:16:49 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit dbb3900775
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 18:09:27 2012 +0800
Fixes for Python 3 compatibility.
commit dfd72131d8
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 15:54:30 2012 +0800
Added protection against proxying swapped models.
commit abcb027190
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 15:11:10 2012 +0800
Cleanup and documentation of AbstractUser base class.
commit a9491a8776
Merge: fd8bb4e08bcb4a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 14:46:49 2012 +0800
Merge commit '08bcb4aec1ed154cefc631b8510ee13e9af0c19d' into t3011
commit fd8bb4e3e4
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 14:20:14 2012 +0800
Documentation improvements coming from community review.
commit b550a6d06d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:52:47 2012 +0800
Refactored skipIfCustomUser into the contrib.auth tests.
commit 52a02f1110
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:46:10 2012 +0800
Refactored common 'get' pattern into manager method.
commit b441a6bbc7
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:41:33 2012 +0800
Added note about backwards incompatible change to admin login messages.
commit 08bcb4aec1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Sep 15 18:30:33 2012 +0300
Splitted User to AbstractUser and User
commit d9f5e5addb
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Sep 15 18:30:02 2012 +0300
Reworked REQUIRED_FIELDS + create_user() interaction
commit 579f152e4a
Merge: 918497293e6733
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:18:37 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 918497218c
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:18:19 2012 +0800
Deprecate AUTH_PROFILE_MODULE and get_profile().
commit 334cdfc1bb
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:00:12 2012 +0800
Added release notes for new swappable User feature.
commit 5d7bb22e8d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 19:59:49 2012 +0800
Ensure swapped models can't be queried.
commit 57ac6e3d32
Merge: f2ec915abfba3b
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 14:31:54 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit f2ec915b20
Merge: 19526565e99a3d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 08:29:51 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 19526563b5
Merge: 2c5e833c4aa26a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 08:22:26 2012 +0800
Merge recent changes from master.
commit 2c5e833a30
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 07:53:46 2012 +0800
Corrected admin_views tests following removal of the email fallback on admin logins.
commit 20d1892491
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 01:00:37 2012 +0800
Added conditional skips for all tests dependent on the default User model
commit 40ea8b8882
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 8 23:47:02 2012 +0800
Added documentation for REQUIRED_FIELDS in custom auth.
commit e6aaf65970
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 8 23:20:02 2012 +0800
Added first draft of custom User docs.
Thanks to Greg Turner for the initial text.
commit 75118bd242
Author: Thomas Sutton <me@thomas-sutton.id.au>
Date: Mon Aug 20 11:17:26 2012 +0800
Admin app should not allow username discovery
The admin app login form should not allow users to discover the username
associated with an email address.
commit d088b3af58
Author: Thomas Sutton <me@thomas-sutton.id.au>
Date: Mon Aug 20 10:32:13 2012 +0800
Admin app login form should use swapped user model
commit 7e82e83d67
Merge: e29c01039aa890
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Fri Sep 7 23:45:03 2012 +0800
Merged master changes.
commit e29c010beb
Merge: 8e3fd7030bdf22
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Aug 20 13:12:57 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 8e3fd703d0
Merge: 507bb5026e0ba0
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Aug 20 13:09:09 2012 +0800
Merged recent changes from trunk.
commit 507bb50a92
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 20:41:37 2012 +0800
Modified auth app so that login with alternate auth app is possible.
commit dabe362836
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 20:10:51 2012 +0800
Modified auth management commands to handle custom user definitions.
commit 7cc0baf89d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 14:17:28 2012 +0800
Added model Meta option for swappable models, and made auth.User a swappable model
* Renamed smart_unicode to smart_text (but kept the old name under
Python 2 for backwards compatibility).
* Renamed smart_str to smart_bytes.
* Re-introduced smart_str as an alias for smart_text under Python 3
and smart_bytes under Python 2 (which is backwards compatible).
Thus smart_str always returns a str objects.
* Used the new smart_str in a few places where both Python 2 and 3
want a str.
This was recently fixed for one-to-one relations; this patch adds
support for foreign keys. Thanks kaiser.yann for the report and
the initial version of the patch.
Many thanks to akaariai for lots of review and feedback, bug finding,
additional unit tests and performance testing.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16930 bcc190cf-cafb-0310-a4f2-bffc1f526a37
The related manager classes are defined within functions, and the methods
had inconsistent and confusing usage of closures vs. parameters on self to
retrieve needed information. Everything is stored on self now.
Also some methods were not using super() where they should have been.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16913 bcc190cf-cafb-0310-a4f2-bffc1f526a37
In r16679 a test was added to verify something had been fixed when a
unicode string type was passed in as a model name. The name has to be
ASCII convertible, but in Python 2.6 and earlier, it must also have str
type.
This commit fixes the problem for earlier Python versions and is almost
identical to a patch from b.leskes in #6045.
Fixes#16689. Refs #6045.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16682 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This is a BACKWARDS INCOMPATIBLE CHANGE for anyone using the signal names introduced in r12223.
* If you were listening to "add", you should now listen to "post_add".
* If you were listening to "remove", you should now listen to "post_remove".
* If you were listening to "clear", you should now listen to "pre_clear".
You may also want to examine your code to see whether the "pre_add", "pre_remove" or "post_clear" would be better suited to your application.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12888 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Alex Gaynor
Russell Keith-Magee
Andrew Godwin
Anssi Kääriäinen
Aymeric Augustin
Jonas Gastal
Loic Bistuer
Simon Charette
Baptiste Mispelon
Alasdair Nicol
Tim Graham
ersran9
Dan Loewenherz
Tim Graham
Ramiro Morales
Mike Fogel
Alex Gaynor
Tomek Paczkowski
Mike Johnson
Jann Kleen
Luke Plant
Claude Paroz
Jannis Leidel
Malcolm Tredinnick
Carl Meyer
Joseph Kocherhans
Adrian Holovaty