When calling model_to_dict, improve performance of the generated SQL by
using values_list to determine primary keys of many to many objects. Add
a specific test for this function, test_model_to_dict_many_to_many
Thanks to brian for the original report and suggested fix.
Ensured that special characters passed in to reverse via the
prefix argument are properly escaped so that calls to
django.utils.regex_helpers.normalize and/or string formatting
operations don't result in exceptions.
Thanks to toofishes for the error report.
This is the start of a deprecation path for the depth kwarg on
select_related. Removing this will allow us to update select_related so
it chains properly and have an API similar to prefetch_related.
Thanks to Marc Tamlyn for spearheading and initial patch.
refs #16855
The dupe avoidance logic was removed as it doesn't seem to do anything,
it is complicated, and it has nearly zero documentation.
The removal of dupe_avoidance allowed for refactoring of both the
implementation and signature of Query.join(). This refactoring cascades
again to some other parts. The most significant of them is the changes
in qs.combine(), and compiler.select_related_descent().
If the current sessions user was logged in via a remote user backend log out
the user if REMOTE_USER header not available - otherwise leave it to other auth
middleware to install the AnonymousUser.
Thanks to Sylvain Bouchard for the initial patch and ticket maintenance.
loads has no way to tell if it should provide text or bytes to the
serializer; bytes are more reasonnable for a serialized representation,
and are the only option for pickled data.
dumps can perform conversions on the value it receives from the
serializer; but for consistency it seems better to require bytes too.
The current code would cause an exception when loading pickled session
data. See next commit.
Also fixed a bug when checking for compressed data.
There was a potential data-loss issue involved -- when clearing
instance's m2m assignments it was possible some other instance's
m2m data was deleted instead.
This commit also improved None handling for to_field cases.
* Prevented stale session files from being loaded
* Added removal of stale session files in django-admin.py clearsessions
Thanks ej for the report, crodjer and Elvard for their inputs.
These tests are expected to fail for the file session backend because it
doesn't handle expiry properly. They didn't because of an error in the
test setup sequence.
Refs #19200, #18194.
This change allows for cleaner tests: we can test the exact output.
Refs #18194: this change makes it possible to compute session expiry
dates at times other than when the session is saved.
Fixed#18458: the existence of the `modification` kwarg implies that you
must pass it to get_expiry_age/date if you call these functions outside
of a short request - response cycle (the intended use case).
The Query.select and Query.select_fields were collapsed into one list
because the attributes had to be always in sync. Now that they are in
one attribute it is impossible to edit them out of sync.
Similar collapse was done for Query.related_select_cols and
Query.related_select_fields.
There was a bug introduced in #18676 which caused fast-path deletes
implemented as "DELETE WHERE pk IN <subquery>" to fail if the SELECT
clause contained additional stuff (for example extra() and annotate()).
Thanks to Trac alias pressureman for spotting this regression.
This case was originally designed to handle Exception's which didn't gracefully support coercing themselves to unicode. However, because it lives in the `else` case of `if hasattr(s, '__unicode__'):` we can be sure it's no longer reachable in djanog anymore, because since Python 2.5 exception has subclassed object, which means Exception objects always have an __unicode__ method.
We probably historically customized it for good reasons, but
currently, the differences with upstream Python are not
significant any longer.
Also fixes#19075 for which a test has been added.
When caching was activated, test_simple_sitemap would fail
because the test result was fetched from cache.
Thanks lucho for the initial patch and krzysiumed@gmail.com for
the review.
Done for consistency with Q() expressions and QuerySet combining. This
will allow usage of '&' and '|' as boolean logical operators in the
future. Refs #16211.
RETURNING is an extension of the SQL standard, which is not implemented
the same by all databases. Allow DatabaseOperations.return_insert_id to
return a None to allow for other 3rd party backends with a different
implementation.
The first try to solve the Python 3 GIS encoding/decoding issue
was too naive. Using decode() on all read strings is bound to fail
as soon as a non-ascii string is concerned.
This patch is a little more clever, leaving ascii decoding when
plain ascii strings are expected, and allowing to specify a custom
encoding in DataSource hierarchy.
This is especially needed to compare XML when hash randomization
is on, as attribute order may vary. Refs #17758, #19038.
Thanks Taylor Mitchell for the initial patch, and Ian Clelland for
review and cleanup.
Previously, a duplicate model, codename for permission would lead to
database integrity error. Cleaned the implementation so that this case
now raises an CommandError instead.
Thanks ckarrie for the report and the initial patches, Flavio Curella
for updating the patch, and Anssi Kääriäinen for testing. See ticket
for other valuable contributors.
Filters at logger level are only processed for messages directly
logged to the specific logger, not for loggers in the parent chain.
As the 'django' logger is almost always processed as an inherited
logger, it makes more sense to filter messages at the 'console'
handler level.
Objects can be fast-path deleted if there are no signals, and there are
no further cascades. If fast-path is taken, the objects do not need to
be loaded into memory before deletion.
Thanks to Jeremy Dunck, Simon Charette and Alex Gaynor for reviewing
the patch.
Thanks to the many people that contributed to the development and review of
this patch, including (but not limited to) Jacob Kaplan-Moss, Anssi
Kääriäinen, Ramiro Morales, Preston Holmes, Josh Ourisman, Thomas Sutton,
and Roger Barnes, as well as the many, many people who have contributed to
the design discussion around this ticket over many years.
Squashed commit of the following:
commit d84749a0f0
Merge: 531e7717c11b1a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Wed Sep 26 18:37:04 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 531e7715da
Merge: 29d1abb1f84b04
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Wed Sep 26 07:09:23 2012 +0800
Merged recent trunk changes.
commit 29d1abbe35
Merge: 8a527dd54c81a1
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 24 07:49:46 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 8a527dda13
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 24 07:48:05 2012 +0800
Ensure sequences are reset correctly in the presence of swapped models.
commit e2b6e22f29
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 17:53:05 2012 +0800
Modifications to the handling and docs for auth forms.
commit 98aba856b5
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 15:28:57 2012 +0800
Improved error handling and docs for get_user_model()
commit 0229209c84
Merge: 6494bf98599f64
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 23 14:50:11 2012 +0800
Merged recent Django trunk changes.
commit 6494bf91f2
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 17 21:38:44 2012 +0800
Improved validation of swappable model settings.
commit 5a04cde342
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Sep 17 07:15:14 2012 +0800
Removed some unused imports.
commit ffd535e413
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 20:31:28 2012 +0800
Corrected attribute access on for get_by_natural_key
commit 913e1ac84c
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 20:12:34 2012 +0800
Added test for proxy model safeguards on swappable models.
commit 280bf19e94
Merge: dbb3900935a863
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 18:16:49 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit dbb3900775
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 18:09:27 2012 +0800
Fixes for Python 3 compatibility.
commit dfd72131d8
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 15:54:30 2012 +0800
Added protection against proxying swapped models.
commit abcb027190
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 15:11:10 2012 +0800
Cleanup and documentation of AbstractUser base class.
commit a9491a8776
Merge: fd8bb4e08bcb4a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 14:46:49 2012 +0800
Merge commit '08bcb4aec1ed154cefc631b8510ee13e9af0c19d' into t3011
commit fd8bb4e3e4
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 14:20:14 2012 +0800
Documentation improvements coming from community review.
commit b550a6d06d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:52:47 2012 +0800
Refactored skipIfCustomUser into the contrib.auth tests.
commit 52a02f1110
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:46:10 2012 +0800
Refactored common 'get' pattern into manager method.
commit b441a6bbc7
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 16 13:41:33 2012 +0800
Added note about backwards incompatible change to admin login messages.
commit 08bcb4aec1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Sep 15 18:30:33 2012 +0300
Splitted User to AbstractUser and User
commit d9f5e5addb
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Sep 15 18:30:02 2012 +0300
Reworked REQUIRED_FIELDS + create_user() interaction
commit 579f152e4a
Merge: 918497293e6733
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:18:37 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 918497218c
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:18:19 2012 +0800
Deprecate AUTH_PROFILE_MODULE and get_profile().
commit 334cdfc1bb
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 20:00:12 2012 +0800
Added release notes for new swappable User feature.
commit 5d7bb22e8d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 19:59:49 2012 +0800
Ensure swapped models can't be queried.
commit 57ac6e3d32
Merge: f2ec915abfba3b
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 15 14:31:54 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit f2ec915b20
Merge: 19526565e99a3d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 08:29:51 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 19526563b5
Merge: 2c5e833c4aa26a
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 08:22:26 2012 +0800
Merge recent changes from master.
commit 2c5e833a30
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 07:53:46 2012 +0800
Corrected admin_views tests following removal of the email fallback on admin logins.
commit 20d1892491
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sun Sep 9 01:00:37 2012 +0800
Added conditional skips for all tests dependent on the default User model
commit 40ea8b8882
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 8 23:47:02 2012 +0800
Added documentation for REQUIRED_FIELDS in custom auth.
commit e6aaf65970
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Sat Sep 8 23:20:02 2012 +0800
Added first draft of custom User docs.
Thanks to Greg Turner for the initial text.
commit 75118bd242
Author: Thomas Sutton <me@thomas-sutton.id.au>
Date: Mon Aug 20 11:17:26 2012 +0800
Admin app should not allow username discovery
The admin app login form should not allow users to discover the username
associated with an email address.
commit d088b3af58
Author: Thomas Sutton <me@thomas-sutton.id.au>
Date: Mon Aug 20 10:32:13 2012 +0800
Admin app login form should use swapped user model
commit 7e82e83d67
Merge: e29c01039aa890
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Fri Sep 7 23:45:03 2012 +0800
Merged master changes.
commit e29c010beb
Merge: 8e3fd7030bdf22
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Aug 20 13:12:57 2012 +0800
Merge remote-tracking branch 'django/master' into t3011
commit 8e3fd703d0
Merge: 507bb5026e0ba0
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Aug 20 13:09:09 2012 +0800
Merged recent changes from trunk.
commit 507bb50a92
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 20:41:37 2012 +0800
Modified auth app so that login with alternate auth app is possible.
commit dabe362836
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 20:10:51 2012 +0800
Modified auth management commands to handle custom user definitions.
commit 7cc0baf89d
Author: Russell Keith-Magee <russell@keith-magee.com>
Date: Mon Jun 4 14:17:28 2012 +0800
Added model Meta option for swappable models, and made auth.User a swappable model
Thanks kmike for the report and initial patch for the changelist->edit
object view link URL.
Other affected links include the delete object one and object history
one (in this case the change had been implemented in commit 5a9e127, this
commit adds admin-quoting of the object PK in a way similar to a222d6e.)
Refs #15294.
Other import errors than ImportError can happen during import of
GDAL files (e.g. OGRException). Some further auditing may be needed
if we want to restrict the catched exceptions at a later stage.
Thanks Ramiro Morales for raising the issue.
Even if buffer and memoryview are not strictly identical, it should
be safe to consider them equivalent for GIS support.
Thanks Aymeric Augustin for the review.
Thanks to Jeremy Dunck for pointing out the problem with this change. If in a
single transaction, the master deletes a record and then get_or_creates a
similar record, under the new behavior the get_or_create would find the record
in the slave db and fail to re-create it, leaving the record nonexistent, which
violates the contract of get_or_create that the record should always exist
afterwards. We need to do everything against the master here in order to ensure
correctness.
This reverts commit 901af86550.
When doing deeper than one level select_related() + only queries(), the
code introduced in b6c356b7bb errored
incorrectly.
Thanks to mrmachine for report & test case.
The display of the ReadOnlyPasswordHashWidget has also been improved to
distinguish empty/unusable password from erroneous password.
Fixed#18453 also.
Thanks danielr and Leo for the reports and Moritz Sichert for the
initial patch.
The problem description in #18239 asserted that
http://bugs.python.org/issue670664 was fixed in Python 2.6.8, but based on
http://bugs.python.org/issue670664#msg146770 it appears that's not correct; the
fix was only applied in 2.7, 3.2, and Python trunk. Therefore we must use our
patched HTMLParser subclass in all Python 2.6 versions.
In an ideal world, nothing except django.db.models.query should have to
import stuff from django.models.sql.*. A few things were needing to get
hold of sql.constants.LOOKUP_SEP, so this commit moves it up to
django.db.models.constants.LOOKUP_SEP.
There are still a couple of places (admin) poking into sql.* to get
QUERY_TERMS, which is unfortunate, but a slightly different issue and
harder to adjust.
Some backends issue a warning here, others not. This is not the primary
goal of the test, so the assertion about the warning has been removed.
Thanks Carl Meyer for noticing the issue and suggesting the fix.
During the new-admin changes, catching of AttributeError was added to
the admin. This patch removes that as it's no longer possible to add a
value to a ModelAdmin that is not available. Adding an attribute that
can not be called causes an ImproperlyConfigured exception to be raised.
Previous this used Python's builtin hash() function, which has never been guarnteed to be stable across implementations (CPython/Jython/etc.) or 32/64 bitness. However, this in practice it was stable. However, with the impending release of Python 3.3 hash randomizations is enabled by default, which would mean the index name changed between program invocations.
Unlike other model fields, the newly introduced (1.4)
GenericIPAddressField did not accept verbose_name and name as the
first positional arguments. This commit fixes it.
Thanks Dan McGee for the report and the patch.
The joins for nested nullable foreign keys were often created as INNER
when they should have been OUTER joins. The reason was that only the
first join in the chain was promoted correctly. There were also issues
with select_related etc.
The basic structure for this problem was:
A -[nullable]-> B -[nonnull]-> C
And the basic problem was that the A->B join was correctly LOUTER,
the B->C join not.
The major change taken in this patch is that now if we promote a join
A->B, we will automatically promote joins B->X for all X in the query.
Also, we now make sure there aren't ever join chains like:
a LOUTER b INNER c
If the a -> b needs to be LOUTER, then the INNER at the end of the
chain will cancel the LOUTER join and we have a broken query.
Sebastian reported this problem and did also major portions of the
patch.
The ORM generated a query with INNER JOIN instead of LEFT OUTER JOIN
in a somewhat complicated case. The main issue was that there was a
chain of nullable FK -> non-nullble FK, and the join promotion logic
didn't see the need to promote the non-nullable FK even if the
previous nullable FK was already promoted to LOUTER JOIN. This resulted
in a query like a LOUTER b INNER c, which incorrectly prunes results.
Made two-digit hours and minutes mandatory in tzinfo (the code used
to crash if a one-digit representation was provided).
Added standalone tests for django.utils.dateparse.
Refs #18764.
Reverted 536b030363 and switched to a more explicit way of avoiding
calling bytes(<int>).
This definitely deserves a refactoring. Specifically, _get_content
should just return b''.join(self). Unfortunately that's impossible
with the current tests.
Remove the params variable from the context and just put the variables
in directly.
This had not been committed previously as the original pattern was used
in the functional generic views and we wanted consistency between them,
but django.views.generic.simple.direct_to_template is now gone so we can
do it 'right'.
Patch by @jphalip updated to apply, documentation and release notes
added.
I've documented strip_tags as well as remove_tags as the difference
between the two wouldn't be immediately obvious.
Python 3 adds a new pickle protocol. The tests were updated in a way
that makes it easy to add another pickle protocol, should Python gain
one.
Thanks Thomas Pelletier for providing an initial version of this patch.
Under some versions of OS X, failure in getting the default system
locale during the syncdb operation of the auth app were causing hard to
diagnose problems afterwards.
No solution based on getpreferredencoding() was chosen because it has
its own problems with certain combinations of Python and OS X versions
(e.g. http://bugs.python.org/issue6202).
Thanks prestonsimmons for the report and prestonsimmons and willhardy
for the initial patch.
Fixes#16017.
In some cases (notably Python 3), when handle_uncaught_exception was
itself raising an exception, the got_request_exception was storing
the latter exception instead of the original exception.
Deferred models now automatically update only the fields which are
loaded from the db (with .only() or .defer()). In addition, any field
set manually after the load is updated on save.
This patch removes an unconditional float(value) conversion from db
backend default convert_values() method. This can cause problems when
aggregating over character fields for example. In addition, Oracle
and SQLite already return the bare value from their convert_values().
In the long term the converting should be done by fields, and the
fields should then call database backend specific converters when
needed. The current setup is inflexible for 3rd party fields.
Thanks to Merlijn van Deen for the original patch.
* Renamed the __unicode__ methods
* Applied the python_2_unicode_compatible decorator
* Removed the StrAndUnicode mix-in that is superseded by
python_2_unicode_compatible
* Kept the __unicode__ methods in classes that specifically
test it under Python 2
This doesn't deal with classes that define both __unicode__ and
__str__; the definition of __str__ should be removed first. It
doesn't guarantee that __str__ will return a str (rather than bytes)
under Python 3 either.
In Python 3, the str type has an __iter__ attribute. Therefore, the
presence of an __iter__ attribute is not sufficient to distinguish
'standard' iterables (list, tuple) from strings.
* Renamed smart_unicode to smart_text (but kept the old name under
Python 2 for backwards compatibility).
* Renamed smart_str to smart_bytes.
* Re-introduced smart_str as an alias for smart_text under Python 3
and smart_bytes under Python 2 (which is backwards compatible).
Thus smart_str always returns a str objects.
* Used the new smart_str in a few places where both Python 2 and 3
want a str.
Make UserCreationForm password validation similar to
SetPasswordForm and AdminPasswordChangeForm, so as the match
check is only done when both passwords are supplied.
Thanks Mitar for the suggestion.
cleaned_data is no longer deleted when form validation fails but only
contains the data that did validate.
Thanks to the various contributors to this patch (see ticket).
The keys/items/values methods return iterators in Python 3, and the
iterkeys/items/values methods don't exist in Python 3. The behavior
under Python 2 is unchanged.
It was broken by a seemingly innocuous change in MultiValueDict.
Document the pitfall for now. This is fragile and should be
considered for refactoring.
Previously, the flush was done before the test case execution and now
it is performed after it.
Other changes to the testing infrastructure include:
* TransactionTestCase now doesn't reset autoincrement sequences either
(previous behavior can achieved by using `reset_sequences`.)
With this, no implicit such reset is performed by any of the provided
TestCase classes.
* New ordering of test cases: All unittest tes cases are run first and
doctests are run at the end.
THse changes could be backward-incompatible with test cases that relied
on some kind of state being preserved between tests. Please read the
relevant sections of the release notes and testing documentation for
further details.
Thanks Andreas Pelme for the initial patch. Karen Tracey and Anssi
Kääriäinen for the feedback and Anssi for reviewing.
This also fixes#12408.
The piece of state is DB connections' allow_thread_sharing attribute
which gets munged test are run when in-memory SQLite databases.
Thanks Anssi for suggesting the possible root cause and Julien for
implementing the fix.
In particular, allow the '--' sequence to be present in string
values without being interpreted as comment marker.
Thanks Tim Chase for the report and shaleh for the initial patch.
Anton I. Sipos
Preston Holmes
Alex Gaynor
Mike Johnson
Alex Gaynor
Aymeric Augustin
Gabriel Hurley
Claude Paroz
Ulrich Petri
Anssi Kääriäinen
Tim Graham
Kent Hauser
Andrew Godwin
Russell Keith-Magee
Luke Plant
Chris McDonald
Ramiro Morales
Ian Clelland
Tom Christie
Carl Meyer
Jan Bednařík
Adrian Holovaty
Mitar
Ludovic Delaveau
Daniele Procida
Brian Galey
Ian Clelland
Justin Bronn
Michael Manfre
Mateusz Haligowski
Stephen Burrows
Michael Farrell
Flavio Curella
Florian Apolloner
Julien Phalip
Adrien Lemaire
Dan Loewenherz
Dave Hall
Jeremy Dunck
Collin Anderson
Mike Grouchy
Martey Dodoo
Malcolm Tredinnick
Travis Swicegood
Travis Swicegood
Florian Apolloner
David Fischer
Simon Meers
Karen Tracey
Marc Tamlyn
Alexey Boriskin
Dmitry Shevchenko
Andrei Antoukh
Andrews Medina
Brendan MacDonell
Justin Bronn
Piet Delport
Jeroen Dekkers