The patch for #19385 caused a regression in certain generic relations
.exclude() filters if a subquery was needed. The fix contains a
refactoring to how Query.split_exclude() and Query.trim_start()
interact.
Thanks to Trac alias nferrari for the report.
Correctly calculate the ``aggregate_start`` offset from loaded fields,
if any are deferred, instead of ``self.query.select`` which includes all
fields on the model.
Also made some PEP 8 fixes.
The SubqueryConstraint defined relabeled_clone(), but that was never
called. Instead there is now clone() and relabel_aliases() methods for
SubqueryConstraint.
A related problem was that SubqueryConstraint didn't correctly use
quote_name_unless_alias() of the outer query. This resulted in failures
when running under PostgreSQL.
When an exception other than IntegrityError was raised, get_or_create
could fail and leave the database connection in an unusable state.
Thanks UloPe for the report.
Used formatted date instead of datetime object for the end of the
year range, as the datetime object loses fractions-of-seconds when
inserted into the db.
This is backward incompatible for custom form field/widgets that rely
on the hard-coded 'Hold down "Control", or "Command" on a Mac, to select
more than one.' sentence.
Application that use standard model form fields and widgets aren't
affected but need to start handling these help texts by themselves
before Django 1.8.
For more details, see the related release notes and deprecation timeline
sections added with this commit.
A regression caused by d5b93d3281 made .get() error
reporting recurse infinitely on certain rare conditions. Fixed this by
not trying to print the given lookup kwargs.
When a GenericRelation was defined on abstract model, queries on childs
of the abstract model didn't work. The problem was in the way fields and
in particular field.rel was copied from models to their children.
The regression was likely caused by #19385. Thanks to Gavin Wahl for
spotting the regression.
A decorator is easier to apply to CBVs. Backwards compatibility isn't an
issue here, except for people running on a recent clone of master.
Fixed a few minor problems in the transactions docs while I was there.
Queries can contain binary data undecodable with utf-8. In this
case, using the 'replace' errors mode when decoding seems like
an acceptable representation of the query.
Thanks Marcel Ryser for the report.
Previously, depending on the database backend or the cursor type,
you'd need to double the percent signs in the query before passing
it to cursor.execute. Now cursor.execute consistently need percent
doubling whenever params argument is not None (placeholder substitution
will happen).
Thanks Thomas Güttler for the report and Walter Doekes for his work
on the patch.
This reverts commit 2cd0edaa47.
This commit was the cause of a memory leak. See ticket for more details.
Thanks Anssi Kääriäinen for identifying the source of the bug.
Model.save() will use UPDATE - if not updated - INSERT instead of
SELECT - if found UPDATE else INSERT. This should save a query when
updating, but will cost a little when inserting model with PK set.
Also fixed#17341 -- made sure .save() commits transactions only after
the whole model has been saved. This wasn't the case in model
inheritance situations.
The save_base implementation was refactored into multiple methods.
A typical chain for inherited save is:
save_base()
_save_parents(self)
for each parent:
_save_parents(parent)
_save_table(parent)
_save_table(self)
Thanks Anssi for haggling until I implemented this.
This change alleviates the need for atomic_if_autocommit. When
autocommit is disabled for a database, atomic will simply create and
release savepoints, and not commit anything. This honors the contract of
not doing any transaction management.
This change also makes the hack to allow using atomic within the legacy
transaction management redundant.
None of the above will work with SQLite, because of a flaw in the design
of the sqlite3 library. This is a known limitation that cannot be lifted
without unacceptable side effects eg. triggering arbitrary commits.
The sql/query.py add_q method did a lot of where/having tree hacking to
get complex queries to work correctly. The logic was refactored so that
it should be simpler to understand. The new logic should also produce
leaner WHERE conditions.
The changes cascade somewhat, as some other parts of Django (like
add_filter() and WhereNode) expect boolean trees in certain format or
they fail to work. So to fix the add_q() one must fix utils/tree.py,
some things in add_filter(), WhereNode and so on.
This commit also fixed add_filter to see negate clauses up the path.
A query like .exclude(Q(reversefk__in=a_list)) didn't work similarly to
.filter(~Q(reversefk__in=a_list)). The reason for this is that only
the immediate parent negate clauses were seen by add_filter, and thus a
tree like AND: (NOT AND: (AND: condition)) will not be handled
correctly, as there is one intermediary AND node in the tree. The
example tree is generated by .exclude(~Q(reversefk__in=a_list)).
Still, aggregation lost connectors in OR cases, and F() objects and
aggregates in same filter clause caused GROUP BY problems on some
databases.
Fixed#17600, fixed#13198, fixed#17025, fixed#17000, fixed#11293.
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
Replaced them with per-database options, for proper multi-db support.
Also toned down the recommendation to tie transactions to HTTP requests.
Thanks Jeremy for sharing his experience.
Since "unless managed" now means "if database-level autocommit",
committing or rolling back doesn't have any effect.
Restored transactional integrity in a few places that relied on
automatically-started transactions with a transitory API.
For users who didn't activate autocommit in their database options, this
is backwards-incompatible in "non-managed" aka "auto" transaction state.
This state now uses database-level autocommit instead of ORM-level
autocommit.
Also removed the uses_autocommit feature which lost its purpose.
Autocommit cannot be manipulated independently from an open connection.
This commit introduces a minor change in behavior: entering transaction
management forces opening a databasse connection. This shouldn't be
backwards incompatible in any practical use case.
enter_transaction_management() was nearly always followed by managed().
In three places it wasn't, but they will all be refactored eventually.
The "forced" keyword argument avoids introducing behavior changes until
then.
This is mostly backwards-compatible, except, of course, for managed
itself. There's a minor difference in _enter_transaction_management:
the top self.transaction_state now contains the new 'managed' state
rather than the previous one. Django doesn't access
self.transaction_state in _enter_transaction_management.
* Grouped related methods together -- with banner comments :/
* Described which methods are intended to be implemented in backends.
* Added docstrings.
* Used the same order in all wrappers.
Squashed commit of the following:
commit 2181d833ed1a2e422494738dcef311164c4e097e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 27 14:28:39 2013 +0100
Fixed#15901 -- Wrapped all PEP-249 exceptions.
commit 5476a5d93c19aa2f928c497d39ce6e33f52694e2
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 17:26:52 2013 +0100
Added PEP 3134 exception chaining.
Thanks Jacob Kaplan-Moss for the suggestion.
commit 9365fad0a650328002fb424457d675a273c95802
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 17:13:49 2013 +0100
Improved API for wrapping database errors.
Thanks Alex Gaynor for the proposal.
commit 1b463b765f2826f73a8d9266795cd5da4f8d5e9e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 15:00:39 2013 +0100
Removed redundant exception wrapping.
This is now taken care of by the cursor wrapper.
commit 524bc7345a724bf526bdd2dd1bcf5ede67d6bb5c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 14:55:10 2013 +0100
Wrapped database exceptions in the base backend.
This covers the most common PEP-249 APIs:
- Connection APIs: close(), commit(), rollback(), cursor()
- Cursor APIs: callproc(), close(), execute(), executemany(),
fetchone(), fetchmany(), fetchall(), nextset().
Fixed#19920.
commit a66746bb5f0839f35543222787fce3b6a0d0a3ea
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 14:53:34 2013 +0100
Added a wrap_database_exception context manager and decorator.
It re-throws backend-specific exceptions using Django's common wrappers.
There were a couple of errors in ._dirty flag handling:
* It started as None, but was never reset to None.
* The _dirty flag was sometimes used to indicate if the connection
was inside transaction management, but this was not done
consistently. This also meant the flag had three separate values.
* The None value had a special meaning, causing for example inability
to commit() on new connection unless enter/leave tx management was
done.
* The _dirty was tracking "connection in transaction" state, but only
in managed transactions.
* Some tests never reset the transaction state of the used connection.
* And some additional less important changes.
This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.
Change patch from https://code.djangoproject.com/ticket/5568
to work on modern Django.
Add special case for MySQL which has different syntax for DROP INDEX.
Add unit tests for the new functionality.
When iteration over a queryset raised an exception, the result cache
remained initialized with an empty list, so subsequent iterations returned
an empty list instead of raising an exception
The regression was likely caused by the fix in #19606 which adjusted
Oracle's unicode detection, though it seems this would have been an
issue in some configurations even before.
Querying the reverse side of nullable to_field relation, where both
sides can contain null values resulted in incorrect results. The reason
was not detecting '' as NULL.
Refs #17541
There was a regression in case two models inherited the same parent,
and one contained a foreign key to other. When select_related travelled
the foreign key the other model reused the parent join made by the
first model. This was likely caused by Query.join_parent_model()
addition in commit 68985db482.
Thanks to Trac alias loic84 for report & tests.
The join promote=True was over-aggressive in select_related handling.
After that was removed, the only other user was query.combine(). That
use case is very easy to handle locally, so there is no more need for
the join(promote=True) flag.
Refs #19849.
This controls whether or not a database level cosntraint is created. This is useful in a few specialized circumstances, but in general should not be used!
The refactoring mainly concentrates on making sure the inner and outer
query agree about the split position. The split position is where the
multijoin happens, and thus the split position also determines the
columns used in the "WHERE col1 IN (SELECT col2 from ...)" condition.
This commit fixes a regression caused by #10790 and commit
69597e5bcc. The regression was caused
by wrong cols in the split position.
Django used to check the version of MySQL before handling the first
request, which required:
- opening a connection
- closing it, to avoid holding it idle until the first request.
This code isn't necessary any longer since Django dropped support for
some versions of MySQL, and other database backends don't implement a
similar dance. For consistency and maintenability, remove it.
Reverts 4423757c0c.
Closes#18135.
Thanks Carl Meyer for the review.
Squashed commit of the following:
commit 4f290bdb60
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 21:21:30 2013 +0100
Used '0:00' instead of 'UTC' which doesn't always exist in Oracle.
Thanks Ian Kelly for the suggestion.
commit 01b6366f3c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 13:38:43 2013 +0100
Made tzname a parameter of datetime_extract/trunc_sql.
This is required to work around a bug in Oracle.
commit 924a144ef8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 14:47:44 2013 +0100
Added support for parameters in SELECT clauses.
commit b4351d2890
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 22:30:22 2013 +0100
Documented backwards incompatibilities in the two previous commits.
commit 91ef84713c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:42:31 2013 +0100
Used QuerySet.datetimes for the admin's date_hierarchy.
commit 0d0de288a5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:29:38 2013 +0100
Used QuerySet.datetimes in date-based generic views.
commit 9c0859ff7c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:25 2013 +0100
Implemented QuerySet.datetimes on Oracle.
commit 68ab511a4f
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:14 2013 +0100
Implemented QuerySet.datetimes on MySQL.
commit 22d52681d3
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:42:29 2013 +0100
Implemented QuerySet.datetimes on SQLite.
commit f6800fd04c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:03 2013 +0100
Implemented QuerySet.datetimes on PostgreSQL.
commit 0c829c23f4
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:41:08 2013 +0100
Added datetime-handling infrastructure in the ORM layers.
commit 104d82a777
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 10:05:55 2013 +0100
Updated null_queries tests to avoid clashing with the __second lookup.
commit c01bbb3235
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 23:07:41 2013 +0100
Updated tests of .dates().
Replaced .dates() by .datetimes() for DateTimeFields.
Replaced dates with datetimes in the expected output for DateFields.
commit 50fb7a5246
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:40:09 2013 +0100
Updated and added tests for QuerySet.datetimes.
commit a8451a5004
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 22:34:46 2013 +0100
Documented the new time lookups and updated the date lookups.
commit 29413eab2b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 16:15:49 2013 +0100
Documented QuerySet.datetimes and updated QuerySet.dates.
The reason was that the except clause needed to remove a connection
from the django.db.connections dict, but other parts of Django do not
expect this to happen. In addition the except clause was silently
swallowing the exception messages.
Refs #19707, special thanks to Carl Meyer for pointing out that this
approach should be taken.
value_annotation isn't very well defined. Before this change, setting it
to datetime.datetime could silently reverse the behavior of isnull
lookups. This commit doesn't have any consequences on the current code.
It's just a safeguard for future ORM hackers.
Previously it was possible to call clear_ordering without the
force_empty argument. The result was that the query was still ordered
by model's meta ordering if that was defined. By making the arg
mandatory it will be easier to spot possible errors caused by assuming
clear_ordering will remove all ordering.
Thanks to Dylan Klomparens for the suggestion. Refs #19720.
When a query had a complex where condition (a condition targeting more
than the base table) a subquery was used for deletion. However, the
query had default ordering from the model's meta and Oracle doesn't
work with ordered subqueries.
The regression was caused by fast-path deletion code introduced in
1cd6e04cd4 for fixing #18676.
Thanks to Dylan Klomparens for the report.
Andrew Godwin
Anssi Kääriäinen
James Aylett
Aymeric Augustin
Dan Loewenherz
Tim Graham
Tim Graham
Tai Lee
Shai Berger
Ramiro Morales
Claude Paroz
Luke Plant
Donald Stufft
Selwin Ong
Erik Romijn
Shai Berger
Mike Fogel
Alex Gaynor
Mike Fogel
Adrian Holovaty
Michael Manfre
Simon Charette
Tobias McNulty
Alex Gaynor
Marti Raudsepp
Gavin Wahl
Marc Tamlyn
Daniele Procida
Florian Apolloner
Loic Bistuer
Honza Král
Tomasz Rybak
Marcin Biernat
Grzegorz Nosek
Tomek Paczkowski