The ORM didn't reuse joins for direct foreign key traversals when using
chained filters. For example:
qs.filter(fk__somefield=1).filter(fk__somefield=2))
produced two joins.
As a bonus, reverse onetoone filters can now reuse joins correctly
The regression was caused by the join() method refactor in commit
68847135bc
Thanks for Simon Charette for spotting some issues with the first draft
of the patch.
This is necessary because get_model() checks are case insensitive, and if the swapable check isn't, the
swappable logic gets tied up in knots with models that are partially swapped out.
Thanks to chris@cogdon.org for the report and extensive analysis, and Preston for his work on the draft patch.
Cleaned up the implementation of base convert_values() a little, and
made sure it accepts None as a value for numeric fields.
There are no tests attached. The reason is that not all of the
convert_values() accept None as a value for numeric fields (for example
sqlite3.convert_values()).
The reason the base convert_values() needs to accept None is that this
situation might arise in custom compilers for 3rd party backends. It
is easy to keep the convert_values() working, so lets do that.
This is a rather large refactoring. The "lookup traversal" code was
splitted out from the setup_joins. There is now names_to_path() method
which does the lookup traveling, the actual work of setup_joins() is
calling names_to_path() and then adding the joins found into the query.
As a side effect it was possible to remove the "process_extra"
functionality used by genric relations. This never worked for left
joins. Now the extra restriction is appended directly to the join
condition instead of the where clause.
To generate the extra condition we need to have the join field
available in the compiler. This has the side-effect that we need more
ugly code in Query.__getstate__ and __setstate__ as Field objects
aren't pickleable.
The join trimming code got a big change - now we trim all direct joins
and never trim reverse joins. This also fixes the problem in #10790
which was join trimming in null filter cases.
The trim argument was used by split_exclude() only to trim the last
join from the given lookup. It is cleaner to just trim the last part
from the lookup in split_exclude() directly so that there is no need
to burden add_filter() with the logic needed for only split_exclude().
This makes it easier to make a ModelBackend-compliant (with regards to
permissions) User model.
Thanks to cdestigter for the report about the relationship between
ModelBackend and permissions, and to the many users on django-dev that
contributed to the discussion about mixins.
ISO formats are automatically appended to the list of input
formats. Kept only when it is specified in first position, as it
has a special meaning in localize_input.
Thanks Bojan Mihelac for the report.
Connection creation was done in db backend ._cursor() call. This
included taking a new connection if needed, initializing the session
state for the new connection and finally creating the connection.
To allow easier modifying of these steps in subclasses (for example to
support connection pools) the _cursor() now calls get_new_connection()
and init_connection_state() if there isn't an existing connection. This
was done for all non-gis core backends. In addition the parameters used
for taking a connection are now created by get_connection_params().
We should also do the same for gis backends and encourage 3rd party
backends to use the same pattern. The pattern is not enforced in code,
and as the backends are private API this will not be required by
documentation either.
F() expressions reuse joins like any lookup in a .filter() call -
reuse multijoins generated in the same .filter() call else generate
new joins. Also, lookups can now reuse joins generated by F().
This change is backwards incompatible, but it is required to prevent
dict randomization from generating different queries depending on
.filter() kwarg ordering. The new way is also more consistent in how
joins are reused.
This allows for Paginator subclasses to easily override the Page class
that gets used.
Took the opportunity to also do some non-invasive PEP8 tidying of the
paginator module.
This allows us to get the trailing slash in URLs without having to
manually assemble and hard-code them.
See also commits e9497a, a222d6, f51eab, 5a9e12.
Capture warnings in Python >= 2.7 and route through
console handler, which is subject to DEBUG==True
Thanks to dstufft for the idea, and claudep for initial patch
Doc cleanup for django.contrib.comments.signals.comment_will_be_sent
If a receiver returns False, an HttpResponse with status code 400
is returned. A test case already exists confirming this behavior.
Updated docs to reflect reality.
Translate before passing in the variables (i.e. close the bracket earlier).
If you translate after passing in the variables the translated value will contain the actual values for 'url' and 'site' and will therefore not be translated.
(I'm not actually using django's flatpages, but I ran into this apparent error when doing a global grep on my own project)
The select_related code got confused when it needed to travel a
reverse relation to a model which had different parent than the
originally travelled relation.
Thanks to Trac aliases shauncutts for report and ungenio for original
patch (committed patch is somewhat modified version of that).
The problem is the same as in #10888 which was reintroduced when
bulk_insert was added. Thanks to Jani Tiainen for report, patch and
also testing the final patch on Oracle GIS.
There was a problem caused by Postgres 9.0+ having bytea_output default
value of 'hex' and cache backend inserting the content as 'bytes' into
a column of type TEXT. Fixed by converting the bytes value to a string
before insert.
Browsers consider the first newline in textareas as some display
artifact, not real content. Hence they are not sending it back to
the server. If we want to keep initial newlines, we have to add one
when we render the textarea.
Thanks bastih for the report and initial patch.
When calling model_to_dict, improve performance of the generated SQL by
using values_list to determine primary keys of many to many objects. Add
a specific test for this function, test_model_to_dict_many_to_many
Thanks to brian for the original report and suggested fix.
Ensured that special characters passed in to reverse via the
prefix argument are properly escaped so that calls to
django.utils.regex_helpers.normalize and/or string formatting
operations don't result in exceptions.
Thanks to toofishes for the error report.
This is the start of a deprecation path for the depth kwarg on
select_related. Removing this will allow us to update select_related so
it chains properly and have an API similar to prefetch_related.
Thanks to Marc Tamlyn for spearheading and initial patch.
refs #16855
The dupe avoidance logic was removed as it doesn't seem to do anything,
it is complicated, and it has nearly zero documentation.
The removal of dupe_avoidance allowed for refactoring of both the
implementation and signature of Query.join(). This refactoring cascades
again to some other parts. The most significant of them is the changes
in qs.combine(), and compiler.select_related_descent().
If the current sessions user was logged in via a remote user backend log out
the user if REMOTE_USER header not available - otherwise leave it to other auth
middleware to install the AnonymousUser.
Thanks to Sylvain Bouchard for the initial patch and ticket maintenance.
loads has no way to tell if it should provide text or bytes to the
serializer; bytes are more reasonnable for a serialized representation,
and are the only option for pickled data.
dumps can perform conversions on the value it receives from the
serializer; but for consistency it seems better to require bytes too.
The current code would cause an exception when loading pickled session
data. See next commit.
Also fixed a bug when checking for compressed data.
There was a potential data-loss issue involved -- when clearing
instance's m2m assignments it was possible some other instance's
m2m data was deleted instead.
This commit also improved None handling for to_field cases.
* Prevented stale session files from being loaded
* Added removal of stale session files in django-admin.py clearsessions
Thanks ej for the report, crodjer and Elvard for their inputs.
These tests are expected to fail for the file session backend because it
doesn't handle expiry properly. They didn't because of an error in the
test setup sequence.
Refs #19200, #18194.
This change allows for cleaner tests: we can test the exact output.
Refs #18194: this change makes it possible to compute session expiry
dates at times other than when the session is saved.
Fixed#18458: the existence of the `modification` kwarg implies that you
must pass it to get_expiry_age/date if you call these functions outside
of a short request - response cycle (the intended use case).
The Query.select and Query.select_fields were collapsed into one list
because the attributes had to be always in sync. Now that they are in
one attribute it is impossible to edit them out of sync.
Similar collapse was done for Query.related_select_cols and
Query.related_select_fields.
There was a bug introduced in #18676 which caused fast-path deletes
implemented as "DELETE WHERE pk IN <subquery>" to fail if the SELECT
clause contained additional stuff (for example extra() and annotate()).
Thanks to Trac alias pressureman for spotting this regression.
This case was originally designed to handle Exception's which didn't gracefully support coercing themselves to unicode. However, because it lives in the `else` case of `if hasattr(s, '__unicode__'):` we can be sure it's no longer reachable in djanog anymore, because since Python 2.5 exception has subclassed object, which means Exception objects always have an __unicode__ method.
We probably historically customized it for good reasons, but
currently, the differences with upstream Python are not
significant any longer.
Also fixes#19075 for which a test has been added.
When caching was activated, test_simple_sitemap would fail
because the test result was fetched from cache.
Thanks lucho for the initial patch and krzysiumed@gmail.com for
the review.
Done for consistency with Q() expressions and QuerySet combining. This
will allow usage of '&' and '|' as boolean logical operators in the
future. Refs #16211.
RETURNING is an extension of the SQL standard, which is not implemented
the same by all databases. Allow DatabaseOperations.return_insert_id to
return a None to allow for other 3rd party backends with a different
implementation.
The first try to solve the Python 3 GIS encoding/decoding issue
was too naive. Using decode() on all read strings is bound to fail
as soon as a non-ascii string is concerned.
This patch is a little more clever, leaving ascii decoding when
plain ascii strings are expected, and allowing to specify a custom
encoding in DataSource hierarchy.
This is especially needed to compare XML when hash randomization
is on, as attribute order may vary. Refs #17758, #19038.
Thanks Taylor Mitchell for the initial patch, and Ian Clelland for
review and cleanup.
Previously, a duplicate model, codename for permission would lead to
database integrity error. Cleaned the implementation so that this case
now raises an CommandError instead.
Thanks ckarrie for the report and the initial patches, Flavio Curella
for updating the patch, and Anssi Kääriäinen for testing. See ticket
for other valuable contributors.
Filters at logger level are only processed for messages directly
logged to the specific logger, not for loggers in the parent chain.
As the 'django' logger is almost always processed as an inherited
logger, it makes more sense to filter messages at the 'console'
handler level.
Objects can be fast-path deleted if there are no signals, and there are
no further cascades. If fast-path is taken, the objects do not need to
be loaded into memory before deletion.
Thanks to Jeremy Dunck, Simon Charette and Alex Gaynor for reviewing
the patch.
Thanks to the many people that contributed to the development and review of
this patch, including (but not limited to) Jacob Kaplan-Moss, Anssi
Kääriäinen, Ramiro Morales, Preston Holmes, Josh Ourisman, Thomas Sutton,
and Roger Barnes, as well as the many, many people who have contributed to
the design discussion around this ticket over many years.
Squashed commit of the following:
commit d84749a0f0
Merge: 531e7717c11b1a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Wed Sep 26 18:37:04 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 531e7715da
Merge: 29d1abb1f84b04
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Wed Sep 26 07:09:23 2012 +0800
Merged recent trunk changes.
commit 29d1abbe35
Merge: 8a527dd54c81a1
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 24 07:49:46 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 8a527dda13
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 24 07:48:05 2012 +0800
Ensure sequences are reset correctly in the presence of swapped models.
commit e2b6e22f29
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 17:53:05 2012 +0800
Modifications to the handling and docs for auth forms.
commit 98aba856b5
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 15:28:57 2012 +0800
Improved error handling and docs for get_user_model()
commit 0229209c84
Merge: 6494bf98599f64
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 14:50:11 2012 +0800
Merged recent Django trunk changes.
commit 6494bf91f2
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 17 21:38:44 2012 +0800
Improved validation of swappable model settings.
commit 5a04cde342
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 17 07:15:14 2012 +0800
Removed some unused imports.
commit ffd535e413
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 20:31:28 2012 +0800
Corrected attribute access on for get_by_natural_key
commit 913e1ac84c
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 20:12:34 2012 +0800
Added test for proxy model safeguards on swappable models.
commit 280bf19e94
Merge: dbb3900935a863
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 18:16:49 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit dbb3900775
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 18:09:27 2012 +0800
Fixes for Python 3 compatibility.
commit dfd72131d8
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 15:54:30 2012 +0800
Added protection against proxying swapped models.
commit abcb027190
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 15:11:10 2012 +0800
Cleanup and documentation of AbstractUser base class.
commit a9491a8776
Merge: fd8bb4e08bcb4a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 14:46:49 2012 +0800
Merge commit '08bcb4aec1ed154cefc631b8510ee13e9af0c19d' into t3011
commit fd8bb4e3e4
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 14:20:14 2012 +0800
Documentation improvements coming from community review.
commit b550a6d06d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:52:47 2012 +0800
Refactored skipIfCustomUser into the contrib.auth tests.
commit 52a02f1110
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:46:10 2012 +0800
Refactored common 'get' pattern into manager method.
commit b441a6bbc7
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:41:33 2012 +0800
Added note about backwards incompatible change to admin login messages.
commit 08bcb4aec1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Sep 15 18:30:33 2012 +0300
Splitted User to AbstractUser and User
commit d9f5e5addb
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Sep 15 18:30:02 2012 +0300
Reworked REQUIRED_FIELDS + create_user() interaction
commit 579f152e4a
Merge: 918497293e6733
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:18:37 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 918497218c
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:18:19 2012 +0800
Deprecate AUTH_PROFILE_MODULE and get_profile().
commit 334cdfc1bb
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:00:12 2012 +0800
Added release notes for new swappable User feature.
commit 5d7bb22e8d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 19:59:49 2012 +0800
Ensure swapped models can't be queried.
commit 57ac6e3d32
Merge: f2ec915abfba3b
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 14:31:54 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit f2ec915b20
Merge: 19526565e99a3d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 08:29:51 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 19526563b5
Merge: 2c5e833c4aa26a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 08:22:26 2012 +0800
Merge recent changes from master.
commit 2c5e833a30
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 07:53:46 2012 +0800
Corrected admin_views tests following removal of the email fallback on admin logins.
commit 20d1892491
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 01:00:37 2012 +0800
Added conditional skips for all tests dependent on the default User model
commit 40ea8b8882
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 8 23:47:02 2012 +0800
Added documentation for REQUIRED_FIELDS in custom auth.
commit e6aaf65970
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 8 23:20:02 2012 +0800
Added first draft of custom User docs.
Thanks to Greg Turner for the initial text.
commit 75118bd242
Author: Thomas Sutton <me@thomas-sutton.id.au>
Date: Mon Aug 20 11:17:26 2012 +0800
Admin app should not allow username discovery
The admin app login form should not allow users to discover the username
associated with an email address.
commit d088b3af58
Author: Thomas Sutton <me@thomas-sutton.id.au>
Date: Mon Aug 20 10:32:13 2012 +0800
Admin app login form should use swapped user model
commit 7e82e83d67
Merge: e29c01039aa890
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Fri Sep 7 23:45:03 2012 +0800
Merged master changes.
commit e29c010beb
Merge: 8e3fd7030bdf22
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Aug 20 13:12:57 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 8e3fd703d0
Merge: 507bb5026e0ba0
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Aug 20 13:09:09 2012 +0800
Merged recent changes from trunk.
commit 507bb50a92
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 20:41:37 2012 +0800
Modified auth app so that login with alternate auth app is possible.
commit dabe362836
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 20:10:51 2012 +0800
Modified auth management commands to handle custom user definitions.
commit 7cc0baf89d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 14:17:28 2012 +0800
Added model Meta option for swappable models, and made auth.User a swappable model
Thanks kmike for the report and initial patch for the changelist->edit
object view link URL.
Other affected links include the delete object one and object history
one (in this case the change had been implemented in commit 5a9e127, this
commit adds admin-quoting of the object PK in a way similar to a222d6e.)
Refs #15294.
Other import errors than ImportError can happen during import of
GDAL files (e.g. OGRException). Some further auditing may be needed
if we want to restrict the catched exceptions at a later stage.
Thanks Ramiro Morales for raising the issue.
Even if buffer and memoryview are not strictly identical, it should
be safe to consider them equivalent for GIS support.
Thanks Aymeric Augustin for the review.
Thanks to Jeremy Dunck for pointing out the problem with this change. If in a
single transaction, the master deletes a record and then get_or_creates a
similar record, under the new behavior the get_or_create would find the record
in the slave db and fail to re-create it, leaving the record nonexistent, which
violates the contract of get_or_create that the record should always exist
afterwards. We need to do everything against the master here in order to ensure
correctness.
This reverts commit 901af86550.
Anssi Kääriäinen
Russell Keith-Magee
Patryk Zawadzki
Tim Graham
Claude Paroz
Baptiste Mispelon
Aymeric Augustin
Ramiro Morales
Florian Apolloner
Andreas Hug
Marc Aymerich
Andrew Godwin
Sebastián Magrí
Preston Holmes
Tom Insam
Julien Phalip
orblivion
Tai Lee
Danilo Bargen
Edward Tjörnhammar
Vladimír Macek
Jonatan Heyman
Chris Khoo
Rigel Di Scala
Flavio Curella
George Hickman
Jannis Leidel
Chris Beaven
Hannes Struss
Riley Strong
Brandon Adams
vanschelven
Adrian Holovaty
Ryan Kaskel
Sean Breant
Alex Gaynor
Anton I. Sipos
Mike Johnson
Alex Gaynor
Gabriel Hurley
Ulrich Petri
Kent Hauser
Luke Plant
Chris McDonald
Ian Clelland
Tom Christie
Carl Meyer
Jan Bednařík
Mitar
Ludovic Delaveau
Daniele Procida
Brian Galey
Ian Clelland
Justin Bronn
Michael Manfre
Mateusz Haligowski
Stephen Burrows
Michael Farrell
Adrien Lemaire
Dan Loewenherz
Dave Hall