The use of OrderedDict (even an empty one) was surprisingly slow. By
initializing OrderedDict only when needed it is possible to save
non-trivial amount of computing time (Model.save() is around 30% faster
for example).
This commit targetted sql.Query only, there are likely other places
which could use similar optimizations.
exc_value might be None even though there's an exception, at least on
Python 2.6. Thanks Thomas Chaumeny for the report.
Fixed#21034.
Forward-port of a8624b2 from 1.6.x.
The option can be used to force pre 1.6 style SELECT on save behaviour.
This is needed in case the database returns zero updated rows even if
there is a matching row in the DB. One such case is PostgreSQL update
trigger that returns NULL.
Reviewed by Tim Graham.
Refs #16649
The __eq__ method now considers two instances without primary key value
equal only when they have same id(). The __hash__ method raises
TypeError for no primary key case.
Fixed#18864, fixed#18250
Thanks to Tim Graham for docs review.
In cases where the same connection (from model A to model B along the
same field) was needed multiple times in a select_related query, the
join setup code mistakenly reused an existing join.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
The bug was already fixed by 01b9c3d519,
so only tests added.
At the same time promote_joins()'s uncoditional flag is gone, it isn't
needed for anything any more.
All Promise objects were passed to force_text() deep in ORM query code.
Not only does this make it difficult or impossible for developers to
prevent or alter this behaviour, but it is also wrong for non-text
fields.
This commit changes `Field.get_prep_value()` from a no-op to one that
resolved Promise objects. All subclasses now call super() method first
to ensure that they have a real value to work with.
The code that tests to see which LIKE expressions to use now runs
using non-error-wrapped cursor, so cx_Oracle exceptions need to be caught
rather than Django DatabaseErrors.
Thanks Trac user ludo for report and initial patch.
It has been possible to use models of wrong type in related field
lookups. For example pigs__in=[a_duck] has worked. Changes to
ForeignObject broke that.
It might be a good idea to restrict the model types usable in lookups.
This should be done intentionally, not accidentally and without any
consideration for deprecation path.
Additionally this patch solves the orthogonal problem that specialized
`QuerySet` like `ValuesQuerySet` didn't inherit from the current `QuerySet`
type. This wasn't an issue until now because we didn't officially support
custom `QuerySet` but it became necessary with the introduction of this new
feature.
Thanks aaugustin, akaariai, carljm, charettes, mjtamlyn, shaib and timgraham
for the reviews.
In the combination of .values().aggregate() the aggregate_select_mask
didn't include the aggregates added. This resulted in bogus query.
Thanks to Trac alias debanshuk for report.
In the intervening years, RelatedField has become less of a hack (though it still is one). Anyone who wants to can re-instate the comment, but please add more details.
Add support for Oracle, fix an issue with the repr of RawQuerySet,
add tests and documentations. Also added a 'supports_paramstyle_pyformat'
database feature, True by default, False for SQLite.
Thanks Donald Stufft for review of documentation.
This is useful:
- to force a rollback on the exit of an atomic block without having to
raise and catch an exception;
- to prevent a rollback after handling an exception manually.
Those methods were only used by `contrib.admin` internally and exclusively
related to `contrib.auth`. Since they were undocumented but used
in the wild the raised deprecation warning point to an also undocumented
alternative that lives in `contrib.auth`.
Also did some PEP8 and other cleanups in the affected modules.
There were a couple of places which used Query.join() directly. By
using setup_joins() in these places the code is more DRY, and in
addition there is no need to directly call field.get_joining_columns()
unless the field is the given join_field from get_path_info(). This
makes it easier to make sure a ForeignObject subclass generates joins
correctly in all cases.
The join used by select_related was incorrectly INNER when the query
had an ORed filter for nullable join that was trimmed away. Fixed this
by forcing the join type to LOUTER even when a join was trimmed away
in ORed queries.
This reverts commit 64041f0e6e.
lookup.tests.LookupTests.test_regex_non_string fails under Postgres.
We should also try to rewrite the test using an existing model.
This can be used to make Django's test suite significantly faster by
reducing the number of models for which content types and permissions
must be created and tables must be flushed in each non-transactional
test.
It's documented for Django contributors and committers but it's branded
as a private API to preserve our freedom to change it in the future.
Most of the credit goes to Anssi. He got the idea and did the research.
Fixed#20483.
This allows using flush on a subset of the tables without having to
manually cascade to all tables with foreign keys to the tables being
truncated, when they're known to be empty.
On databases where truncate is implemented with DELETE FROM, this
doesn't make a difference. The cascade is allowed, not mandatory.
The patch for #19385 caused a regression in certain generic relations
.exclude() filters if a subquery was needed. The fix contains a
refactoring to how Query.split_exclude() and Query.trim_start()
interact.
Thanks to Trac alias nferrari for the report.
Correctly calculate the ``aggregate_start`` offset from loaded fields,
if any are deferred, instead of ``self.query.select`` which includes all
fields on the model.
Also made some PEP 8 fixes.
The SubqueryConstraint defined relabeled_clone(), but that was never
called. Instead there is now clone() and relabel_aliases() methods for
SubqueryConstraint.
A related problem was that SubqueryConstraint didn't correctly use
quote_name_unless_alias() of the outer query. This resulted in failures
when running under PostgreSQL.
When an exception other than IntegrityError was raised, get_or_create
could fail and leave the database connection in an unusable state.
Thanks UloPe for the report.
Used formatted date instead of datetime object for the end of the
year range, as the datetime object loses fractions-of-seconds when
inserted into the db.
This is backward incompatible for custom form field/widgets that rely
on the hard-coded 'Hold down "Control", or "Command" on a Mac, to select
more than one.' sentence.
Application that use standard model form fields and widgets aren't
affected but need to start handling these help texts by themselves
before Django 1.8.
For more details, see the related release notes and deprecation timeline
sections added with this commit.
A regression caused by d5b93d3281 made .get() error
reporting recurse infinitely on certain rare conditions. Fixed this by
not trying to print the given lookup kwargs.
When a GenericRelation was defined on abstract model, queries on childs
of the abstract model didn't work. The problem was in the way fields and
in particular field.rel was copied from models to their children.
The regression was likely caused by #19385. Thanks to Gavin Wahl for
spotting the regression.
A decorator is easier to apply to CBVs. Backwards compatibility isn't an
issue here, except for people running on a recent clone of master.
Fixed a few minor problems in the transactions docs while I was there.
Queries can contain binary data undecodable with utf-8. In this
case, using the 'replace' errors mode when decoding seems like
an acceptable representation of the query.
Thanks Marcel Ryser for the report.
Previously, depending on the database backend or the cursor type,
you'd need to double the percent signs in the query before passing
it to cursor.execute. Now cursor.execute consistently need percent
doubling whenever params argument is not None (placeholder substitution
will happen).
Thanks Thomas Güttler for the report and Walter Doekes for his work
on the patch.
This reverts commit 2cd0edaa47.
This commit was the cause of a memory leak. See ticket for more details.
Thanks Anssi Kääriäinen for identifying the source of the bug.
Model.save() will use UPDATE - if not updated - INSERT instead of
SELECT - if found UPDATE else INSERT. This should save a query when
updating, but will cost a little when inserting model with PK set.
Also fixed#17341 -- made sure .save() commits transactions only after
the whole model has been saved. This wasn't the case in model
inheritance situations.
The save_base implementation was refactored into multiple methods.
A typical chain for inherited save is:
save_base()
_save_parents(self)
for each parent:
_save_parents(parent)
_save_table(parent)
_save_table(self)
Thanks Anssi for haggling until I implemented this.
This change alleviates the need for atomic_if_autocommit. When
autocommit is disabled for a database, atomic will simply create and
release savepoints, and not commit anything. This honors the contract of
not doing any transaction management.
This change also makes the hack to allow using atomic within the legacy
transaction management redundant.
None of the above will work with SQLite, because of a flaw in the design
of the sqlite3 library. This is a known limitation that cannot be lifted
without unacceptable side effects eg. triggering arbitrary commits.
The sql/query.py add_q method did a lot of where/having tree hacking to
get complex queries to work correctly. The logic was refactored so that
it should be simpler to understand. The new logic should also produce
leaner WHERE conditions.
The changes cascade somewhat, as some other parts of Django (like
add_filter() and WhereNode) expect boolean trees in certain format or
they fail to work. So to fix the add_q() one must fix utils/tree.py,
some things in add_filter(), WhereNode and so on.
This commit also fixed add_filter to see negate clauses up the path.
A query like .exclude(Q(reversefk__in=a_list)) didn't work similarly to
.filter(~Q(reversefk__in=a_list)). The reason for this is that only
the immediate parent negate clauses were seen by add_filter, and thus a
tree like AND: (NOT AND: (AND: condition)) will not be handled
correctly, as there is one intermediary AND node in the tree. The
example tree is generated by .exclude(~Q(reversefk__in=a_list)).
Still, aggregation lost connectors in OR cases, and F() objects and
aggregates in same filter clause caused GROUP BY problems on some
databases.
Fixed#17600, fixed#13198, fixed#17025, fixed#17000, fixed#11293.
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
Replaced them with per-database options, for proper multi-db support.
Also toned down the recommendation to tie transactions to HTTP requests.
Thanks Jeremy for sharing his experience.
Since "unless managed" now means "if database-level autocommit",
committing or rolling back doesn't have any effect.
Restored transactional integrity in a few places that relied on
automatically-started transactions with a transitory API.
For users who didn't activate autocommit in their database options, this
is backwards-incompatible in "non-managed" aka "auto" transaction state.
This state now uses database-level autocommit instead of ORM-level
autocommit.
Also removed the uses_autocommit feature which lost its purpose.
Autocommit cannot be manipulated independently from an open connection.
This commit introduces a minor change in behavior: entering transaction
management forces opening a databasse connection. This shouldn't be
backwards incompatible in any practical use case.
enter_transaction_management() was nearly always followed by managed().
In three places it wasn't, but they will all be refactored eventually.
The "forced" keyword argument avoids introducing behavior changes until
then.
This is mostly backwards-compatible, except, of course, for managed
itself. There's a minor difference in _enter_transaction_management:
the top self.transaction_state now contains the new 'managed' state
rather than the previous one. Django doesn't access
self.transaction_state in _enter_transaction_management.
* Grouped related methods together -- with banner comments :/
* Described which methods are intended to be implemented in backends.
* Added docstrings.
* Used the same order in all wrappers.
Squashed commit of the following:
commit 2181d833ed1a2e422494738dcef311164c4e097e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 27 14:28:39 2013 +0100
Fixed#15901 -- Wrapped all PEP-249 exceptions.
commit 5476a5d93c19aa2f928c497d39ce6e33f52694e2
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 17:26:52 2013 +0100
Added PEP 3134 exception chaining.
Thanks Jacob Kaplan-Moss for the suggestion.
commit 9365fad0a650328002fb424457d675a273c95802
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 17:13:49 2013 +0100
Improved API for wrapping database errors.
Thanks Alex Gaynor for the proposal.
commit 1b463b765f2826f73a8d9266795cd5da4f8d5e9e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 15:00:39 2013 +0100
Removed redundant exception wrapping.
This is now taken care of by the cursor wrapper.
commit 524bc7345a724bf526bdd2dd1bcf5ede67d6bb5c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 14:55:10 2013 +0100
Wrapped database exceptions in the base backend.
This covers the most common PEP-249 APIs:
- Connection APIs: close(), commit(), rollback(), cursor()
- Cursor APIs: callproc(), close(), execute(), executemany(),
fetchone(), fetchmany(), fetchall(), nextset().
Fixed#19920.
commit a66746bb5f0839f35543222787fce3b6a0d0a3ea
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Feb 26 14:53:34 2013 +0100
Added a wrap_database_exception context manager and decorator.
It re-throws backend-specific exceptions using Django's common wrappers.
There were a couple of errors in ._dirty flag handling:
* It started as None, but was never reset to None.
* The _dirty flag was sometimes used to indicate if the connection
was inside transaction management, but this was not done
consistently. This also meant the flag had three separate values.
* The None value had a special meaning, causing for example inability
to commit() on new connection unless enter/leave tx management was
done.
* The _dirty was tracking "connection in transaction" state, but only
in managed transactions.
* Some tests never reset the transaction state of the used connection.
* And some additional less important changes.
This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.
Anssi Kääriäinen
Tim Graham
Alex Gaynor
Michael Manfre
Gregor MacGregor
Loic Bistuer
Садовский Николай
Aymeric Augustin
Andrew Godwin
Eric Boersma
Chris Wilson
Carl Meyer
Simon Charette
Claude Paroz
Dima Kurguzov
Harm Geerts
Alex Cucu
Curtis Maloney
Alex Couper
Florian Hahn
Tai Lee
Shai Berger
Julien Phalip
ersran9
Karol Sikora
Ramiro Morales
Andrew Clark
Gilberto Gonçalves
Baptiste Mispelon
Axel Haustant
James Aylett
Dan Loewenherz
Tim Graham
Luke Plant
Donald Stufft
Selwin Ong
Erik Romijn
Shai Berger
Mike Fogel
Mike Fogel
Adrian Holovaty
Tobias McNulty
Alex Gaynor
Marti Raudsepp
Gavin Wahl
Marc Tamlyn
Daniele Procida
Florian Apolloner
Honza Král