Commit Graph

282 Commits

Author SHA1 Message Date
Anssi Kääriäinen d744c550d5 Fixed #19964 -- Removed relabel_aliases from some structs
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
2013-03-12 21:33:47 +02:00
Aymeric Augustin 0c82b1dfc4 Fixed #19929 -- Improved error when MySQL doesn't have TZ definitions.
Thanks tomas_00 for the report.
2013-02-28 17:35:13 +01:00
Anssi Kääriäinen 06de130dae Fixed #12823 -- Was already fixed in master, tests added
Also added a little improvement to sql/query.py to get rid of
non-necessary IS NOT NULL check.
2013-02-28 15:05:48 +02:00
Anssi Kääriäinen 50328f0a61 Fixed #19861 -- Transaction ._dirty flag improvement
There were a couple of errors in ._dirty flag handling:
  * It started as None, but was never reset to None.
  * The _dirty flag was sometimes used to indicate if the connection
    was inside transaction management, but this was not done
    consistently. This also meant the flag had three separate values.
  * The None value had a special meaning, causing for example inability
    to commit() on new connection unless enter/leave tx management was
    done.
  * The _dirty was tracking "connection in transaction" state, but only
    in managed transactions.
  * Some tests never reset the transaction state of the used connection.
  * And some additional less important changes.

This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.
2013-02-27 17:54:27 +02:00
Honza Kral cbb9f629b8 Revert "fixes #19263" - Fails if not SQLite
This reverts commit 2b76f19f2b.
2013-02-24 12:38:32 +01:00
Marcin Biernat 2b76f19f2b fixes #19263 2013-02-23 23:53:46 +01:00
Anssi Kääriäinen 3c6318e831 Fixed #19870 -- Regression in select_related in inheritance cases
There was a regression in case two models inherited the same parent,
and one contained a foreign key to other. When select_related travelled
the foreign key the other model reused the parent join made by the
first model. This was likely caused by Query.join_parent_model()
addition in commit 68985db482.

Thanks to Trac alias loic84 for report & tests.
2013-02-21 12:01:23 +02:00
Anssi Kääriäinen 10f9ba046f Cleaned up join promotion in query.combine() with OR
Refs #19849
2013-02-20 21:45:36 +02:00
Anssi Kääriäinen edf93127bf Removed join() promote kwarg
The join promote=True was over-aggressive in select_related handling.
After that was removed, the only other user was query.combine(). That
use case is very easy to handle locally, so there is no more need for
the join(promote=True) flag.

Refs #19849.
2013-02-20 21:43:44 +02:00
Anssi Kääriäinen 3fef304ff2 Refactored negated IS NULL handling
This one cleaned up add_filter() negated filter generation. As a side
effect split_exclude() was cleaned up, too.

Refs #19849
2013-02-20 21:34:01 +02:00
Anssi Kääriäinen b4492a8ca4 Fixed #19837 -- Refactored split_exclude() join generation
The refactoring mainly concentrates on making sure the inner and outer
query agree about the split position. The split position is where the
multijoin happens, and thus the split position also determines the
columns used in the "WHERE col1 IN (SELECT col2 from ...)" condition.

This commit fixes a regression caused by #10790 and commit
69597e5bcc. The regression was caused
by wrong cols in the split position.
2013-02-19 01:10:49 +02:00
Aymeric Augustin e74e207cce Fixed #17260 -- Added time zone aware aggregation and lookups.
Thanks Carl Meyer for the review.

Squashed commit of the following:

commit 4f290bdb60
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Wed Feb 13 21:21:30 2013 +0100

    Used '0:00' instead of 'UTC' which doesn't always exist in Oracle.

    Thanks Ian Kelly for the suggestion.

commit 01b6366f3c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Wed Feb 13 13:38:43 2013 +0100

    Made tzname a parameter of datetime_extract/trunc_sql.

    This is required to work around a bug in Oracle.

commit 924a144ef8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Wed Feb 13 14:47:44 2013 +0100

    Added support for parameters in SELECT clauses.

commit b4351d2890
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Mon Feb 11 22:30:22 2013 +0100

    Documented backwards incompatibilities in the two previous commits.

commit 91ef84713c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Mon Feb 11 09:42:31 2013 +0100

    Used QuerySet.datetimes for the admin's date_hierarchy.

commit 0d0de288a5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Mon Feb 11 09:29:38 2013 +0100

    Used QuerySet.datetimes in date-based generic views.

commit 9c0859ff7c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 21:43:25 2013 +0100

    Implemented QuerySet.datetimes on Oracle.

commit 68ab511a4f
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 21:43:14 2013 +0100

    Implemented QuerySet.datetimes on MySQL.

commit 22d52681d3
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 21:42:29 2013 +0100

    Implemented QuerySet.datetimes on SQLite.

commit f6800fd04c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 21:43:03 2013 +0100

    Implemented QuerySet.datetimes on PostgreSQL.

commit 0c829c23f4
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 21:41:08 2013 +0100

    Added datetime-handling infrastructure in the ORM layers.

commit 104d82a777
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Mon Feb 11 10:05:55 2013 +0100

    Updated null_queries tests to avoid clashing with the __second lookup.

commit c01bbb3235
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 23:07:41 2013 +0100

    Updated tests of .dates().

    Replaced .dates() by .datetimes() for DateTimeFields.
    Replaced dates with datetimes in the expected output for DateFields.

commit 50fb7a5246
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 21:40:09 2013 +0100

    Updated and added tests for QuerySet.datetimes.

commit a8451a5004
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 22:34:46 2013 +0100

    Documented the new time lookups and updated the date lookups.

commit 29413eab2b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sun Feb 10 16:15:49 2013 +0100

    Documented QuerySet.datetimes and updated QuerySet.dates.
2013-02-16 09:19:04 +01:00
Aymeric Augustin 7db770b013 Added a check in the creation of IS NULL clauses.
value_annotation isn't very well defined. Before this change, setting it
to datetime.datetime could silently reverse the behavior of isnull
lookups. This commit doesn't have any consequences on the current code.
It's just a safeguard for future ORM hackers.
2013-02-11 10:38:39 +01:00
Anssi Kääriäinen 5cc0f5f8c1 Made Query.clear_ordering force_empty arg mandatory
Previously it was possible to call clear_ordering without the
force_empty argument. The result was that the query was still ordered
by model's meta ordering if that was defined. By making the arg
mandatory it will be easier to spot possible errors caused by assuming
clear_ordering will remove all ordering.

Thanks to Dylan Klomparens for the suggestion. Refs #19720.
2013-02-10 20:58:25 +02:00
Anssi Kääriäinen 23ca3a0194 Fixed #16759 -- Remove use of __deepcopy__ in qs.clone()
The original problem was that queryset cloning was really expensive
when filtering with F() clauses. The __deepcopy__ went too deep copying
_meta attributes of the models used. To fix this the use of
__deepcopy__ in qs cloning was removed.

This commit results in some speed improvements across the djangobench
benchmark suite. Most query_* tests are 20-30% faster, save() is 50%
faster and finally complex filtering situations can see 2x to order
of magnitude improvments.

Thanks to Suor, Alex and lrekucki for valuable feedback.
2013-01-08 19:17:13 +02:00
Anssi Kääriäinen a2396a4c8f Fixed #19173 -- Made EmptyQuerySet a marker class only
The guarantee that no queries will be made when accessing results is
done by new EmptyWhere class which is used for query.where and having.

Thanks to Simon Charette for reviewing and valuable suggestions.
2013-01-06 19:18:28 +02:00
Anssi Kääriäinen ce3c71faf1 Minor improvement to proxy model handling
Refs #19385
2012-12-30 11:20:27 +02:00
Anssi Kääriäinen 807eff7439 Made use of PathInfo.direct flag in trim_joins
Refs #19385
2012-12-30 11:20:13 +02:00
Anssi Kääriäinen 68985db482 Added Query.join_parent_model()
This simplifies especially compiler.py a lot, where almost the same
code was repeated multiple times.

Refs #19385
2012-12-30 11:19:35 +02:00
Anssi Kääriäinen 4511aeb6b8 Moved join path generation to Field
Refs #19385
2012-12-30 11:19:19 +02:00
Anssi Kääriäinen 9ef3cab40b Made sure join_field is always available in .join()
Refs #19385
2012-12-30 11:18:25 +02:00
Anssi Kääriäinen 4007c8f6eb Fixed a regression in distinct_on
Caused by regression fix for #19500.
2012-12-20 22:50:06 +02:00
Anssi Kääriäinen d407164c04 Fixed #18854 -- Join promotion in disjunction cases
The added promotion logic is based on promoting any joins used in only
some of the childs of an OR clause unless the join existed before the
OR clause addition.
2012-12-20 21:45:15 +02:00
Anssi Kääriäinen 3dcd435a0e Fixed #19500 -- Solved a regression in join reuse
The ORM didn't reuse joins for direct foreign key traversals when using
chained filters. For example:
    qs.filter(fk__somefield=1).filter(fk__somefield=2))
produced two joins.

As a bonus, reverse onetoone filters can now reuse joins correctly

The regression was caused by the join() method refactor in commit
68847135bc

Thanks for Simon Charette for spotting some issues with the first draft
of the patch.
2012-12-20 21:27:00 +02:00
Anssi Kääriäinen c64b57d166 Replaced '__' with LOOKUP_SEP in sql/query.py
Thanks to Simon Charette for report.
2012-12-17 21:15:01 +02:00
Anssi Kääriäinen 69597e5bcc Fixed #10790 -- Refactored sql.Query.setup_joins()
This is a rather large refactoring. The "lookup traversal" code was
splitted out from the setup_joins. There is now names_to_path() method
which does the lookup traveling, the actual work of setup_joins() is
calling names_to_path() and then adding the joins found into the query.

As a side effect it was possible to remove the "process_extra"
functionality used by genric relations. This never worked for left
joins. Now the extra restriction is appended directly to the join
condition instead of the where clause.

To generate the extra condition we need to have the join field
available in the compiler. This has the side-effect that we need more
ugly code in Query.__getstate__ and __setstate__ as Field objects
aren't pickleable.

The join trimming code got a big change - now we trim all direct joins
and never trim reverse joins. This also fixes the problem in #10790
which was join trimming in null filter cases.
2012-12-16 17:23:26 +02:00
Anssi Kääriäinen f811649710 Fixed #18816 -- Removed "trim" argument from add_filter()
The trim argument was used by split_exclude() only to trim the last
join from the given lookup. It is cleaner to just trim the last part
from the lookup in split_exclude() directly so that there is no need
to burden add_filter() with the logic needed for only split_exclude().
2012-12-16 16:56:28 +02:00
Tai Lee 6ebf115206 Fixed #14694 -- Made ``defer()`` work with reverse relations
Reverse o2o fields are now usable with defer.
2012-11-28 18:17:10 +02:00
Anssi Kääriäinen 08b4a22293 Updated stale docstring of setup_joins 2012-11-24 16:26:05 +02:00
Anssi Kääriäinen 90b86291d0 Fixed #18375 -- Removed dict-ordering dependency for F-expressions
F() expressions reuse joins like any lookup in a .filter() call -
reuse multijoins generated in the same .filter() call else generate
new joins. Also, lookups can now reuse joins generated by F().

This change is backwards incompatible, but it is required to prevent
dict randomization from generating different queries depending on
.filter() kwarg ordering. The new way is also more consistent in how
joins are reused.
2012-11-23 19:53:04 +02:00
Anssi Kääriäinen f51e409a5f Fixed #13781 -- Improved select_related in inheritance situations
The select_related code got confused when it needed to travel a
reverse relation to a model which had different parent than the
originally travelled relation.

Thanks to Trac aliases shauncutts for report and ungenio for original
patch (committed patch is somewhat modified version of that).
2012-11-15 17:15:21 +02:00
Anssi Kääriäinen 92d7f541da Fixed #19058 -- Fixed Oracle GIS crash
The problem is the same as in #10888 which was reintroduced when
bulk_insert was added. Thanks to Jani Tiainen for report, patch and
also testing the final patch on Oracle GIS.
2012-11-15 16:08:06 +02:00
Anssi Kääriäinen ce1af8d702 Removed use of SortedDict for query.alias_refcount
This will have a smallish impact on performance. Refs #19276.
2012-11-13 22:48:25 +02:00
Anssi Kääriäinen cafb266954 Fixed #17144 -- MySQL again groups by PK only
Thanks to Christian Oudard for the report and tests.
2012-11-08 00:56:32 +02:00
Anssi Kääriäinen 68847135bc Removed dupe_avoidance from sql/query and sql/compiler.py
The dupe avoidance logic was removed as it doesn't seem to do anything,
it is complicated, and it has nearly zero documentation.

The removal of dupe_avoidance allowed for refactoring of both the
implementation and signature of Query.join(). This refactoring cascades
again to some other parts. The most significant of them is the changes
in qs.combine(), and compiler.select_related_descent().
2012-10-31 08:19:44 +02:00
Anssi Kääriäinen 11699ac4b5 Fixed #19190 -- Refactored Query select clause attributes
The Query.select and Query.select_fields were collapsed into one list
because the attributes had to be always in sync. Now that they are in
one attribute it is impossible to edit them out of sync.

Similar collapse was done for Query.related_select_cols and
Query.related_select_fields.
2012-10-27 02:13:02 +03:00
Luke Plant f3a2bcdee9 Fixed #15040 - Boolean fields return 0 and 1 when loaded through select_related
Thanks to homm for the report and ramiro for the patch.
2012-10-26 00:25:59 +01:00
Anssi Kääriäinen bd6d9ea87c Fixed regression caused by #19102 2012-10-25 19:13:10 +03:00
Anssi Kääriäinen 11b8712cc7 Added docstring to DeleteQuery.delete_qs() 2012-10-25 17:16:56 +03:00
Anssi Kääriäinen f64a5ef404 Fixed #19102 -- Fixed fast-path delete for modified SELECT clause cases
There was a bug introduced in #18676 which caused fast-path deletes
implemented as "DELETE WHERE pk IN <subquery>" to fail if the SELECT
clause contained additional stuff (for example extra() and annotate()).

Thanks to Trac alias pressureman for spotting this regression.
2012-10-25 17:16:44 +03:00
Michael Manfre c2150d4d2c Fixed #19096 -- Made can_return_id_from_insert more extendable
RETURNING is an extension of the SQL standard, which is not implemented
the same by all databases. Allow DatabaseOperations.return_insert_id to
return a None to allow for other 3rd party backends with a different
implementation.
2012-10-10 01:00:58 +03:00
Anssi Kääriäinen a62d53c032 Fixed #19087 -- Ensured query's base table is never LOUTER joined
This fixes a regression created by join promotion logic refactoring:
01b9c3d519

Thanks to Ivan Virabyan for the report.
2012-10-08 18:40:09 +03:00
Anssi Kääriäinen ddd7d1af20 Avoided storing ExpressionNodes in dicts 2012-09-30 17:50:56 +03:00
Anssi Kääriäinen 1cd6e04cd4 Fixed #18676 -- Allow fast-path deletion of objects
Objects can be fast-path deleted if there are no signals, and there are
no further cascades. If fast-path is taken, the objects do not need to
be loaded into memory before deletion.

Thanks to Jeremy Dunck, Simon Charette and Alex Gaynor for reviewing
the patch.
2012-09-28 18:16:08 +03:00
Anssi Kääriäinen f399a804c9 Fixed #17485 regression -- only + select_related interaction
When doing deeper than one level select_related() + only queries(), the
code introduced in b6c356b7bb errored
incorrectly.

Thanks to mrmachine for report & test case.
2012-09-16 22:58:40 +03:00
Malcolm Tredinnick c4aa26a983 Internal refactoring; moving LOOKUP_SEP up one level.
In an ideal world, nothing except django.db.models.query should have to
import stuff from django.models.sql.*. A few things were needing to get
hold of sql.constants.LOOKUP_SEP, so this commit moves it up to
django.db.models.constants.LOOKUP_SEP.

There are still a couple of places (admin) poking into sql.* to get
QUERY_TERMS, which is unfortunate, but a slightly different issue and
harder to adjust.
2012-09-08 19:51:36 -04:00
Alex Gaynor 0e296131bb Cleaned up some small bits of the ORM, including removing an import *. 2012-09-07 10:58:17 -04:00
Mitar b2ffa2ea90 Code comment fix. 2012-08-26 14:28:44 -07:00
Anssi Kääriäinen 01b9c3d519 Fixed #16715 -- Fixed join promotion logic for nested nullable FKs
The joins for nested nullable foreign keys were often created as INNER
when they should have been OUTER joins. The reason was that only the
first join in the chain was promoted correctly. There were also issues
with select_related etc.

The basic structure for this problem was:
  A -[nullable]-> B -[nonnull]-> C

And the basic problem was that the A->B join was correctly LOUTER,
the B->C join not.

The major change taken in this patch is that now if we promote a join
A->B, we will automatically promote joins B->X for all X in the query.
Also, we now make sure there aren't ever join chains like:
   a LOUTER b INNER c
If the a -> b needs to be LOUTER, then the INNER at the end of the
chain will cancel the LOUTER join and we have a broken query.

Sebastian reported this problem and did also major portions of the
patch.
2012-08-25 14:14:45 +03:00
Anssi Kääriäinen a193372753 Fixed #17886 -- Fixed join promotion in ORed nullable queries
The ORM generated a query with INNER JOIN instead of LEFT OUTER JOIN
in a somewhat complicated case. The main issue was that there was a
chain of nullable FK -> non-nullble FK, and the join promotion logic
didn't see the need to promote the non-nullable FK even if the
previous nullable FK was already promoted to LOUTER JOIN. This resulted
in a query like a LOUTER b INNER c, which incorrectly prunes results.
2012-08-21 21:23:57 +03:00