Commit Graph

386 Commits

Author SHA1 Message Date
Rodolfo Carvalho 0d91225892 Fixed many typos in comments and docstrings.
Thanks Piotr Kasprzyk for help with the patch.
2014-03-03 07:38:09 -05:00
Martin Matusiak 48a8b714d4 Fixed bad grammar in a comment. 2014-02-25 21:02:35 -05:00
Anssi Kääriäinen 35cecb1ebd Fixed #21748 -- join promotion for negated AND conditions
Made sure Django treats case .filter(NOT (a AND b)) the same way as
.filter((NOT a OR NOT b)) for join promotion.
2014-02-04 18:48:06 +02:00
Anssi Kääriäinen 78a2617703 Fixed #21787 -- regression in MTI .exclude() queries 2014-01-24 10:10:17 +02:00
Anssi Kääriäinen 994c842bc9 Fixed Oracle query failures caused by lookup refactor
Refs #21821.
2014-01-19 10:09:17 +02:00
Anssi Kääriäinen 20bab2cf9d Fixed #16187 -- refactored ORM lookup system
Allowed users to specify which lookups or transforms ("nested lookus")
are available for fields. The implementation is now class based.

Squashed commit of the following:

commit fa7a7195f1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 18 10:53:24 2014 +0200

    Added lookup registration API docs

commit eb1c8ce164
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Tue Jan 14 18:59:36 2014 +0200

    Release notes and other minor docs changes

commit 11501c29c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Jan 12 20:53:03 2014 +0200

    Forgot to add custom_lookups tests in prev commit

commit 83173b960e
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Jan 12 19:59:12 2014 +0200

    Renamed Extract -> Transform

commit 3b18d9f3a1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Jan 12 19:51:53 2014 +0200

    Removed suggestion of temporary lookup registration from docs

commit 21d0c7631c
Merge: 2509006 f2dc442
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Jan 12 09:38:23 2014 -0800

    Merge pull request #2 from mjtamlyn/lookups_3

    Reworked custom lookups docs.

commit f2dc4429a1
Author: Marc Tamlyn <marc.tamlyn@gmail.com>
Date:   Sun Jan 12 13:15:05 2014 +0000

    Reworked custom lookups docs.

    Mostly just formatting and rewording, but also replaced the example
    using ``YearExtract`` to  use an example which is unlikely to ever be
    possible directly in the ORM.

commit 2509006506
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Jan 12 13:19:13 2014 +0200

    Removed unused import

commit 4fba5dfaa0
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 22:34:41 2014 +0200

    Added docs to index

commit 6d53963f37
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 22:10:24 2014 +0200

    Dead code removal

commit f9cc039007
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 19:00:43 2014 +0200

    A new try for docs

commit 33aa18a6e3
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 14:57:12 2014 +0200

    Renamed get_cols to get_group_by_cols

commit c7d5f8661b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 14:45:53 2014 +0200

    Altered query string customization for backends vendors

    The new way is trying to call first method 'as_' + connection.vendor.
    If that doesn't exist, then call as_sql().

    Also altered how lookup registration is done. There is now
    RegisterLookupMixin class that is used by Field, Extract and
    sql.Aggregate. This allows one to register lookups for extracts and
    aggregates in the same way lookup registration is done for fields.

commit 90e7004ec1
Merge: 66649ff f7c2c0a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 13:21:01 2014 +0200

    Merge branch 'master' into lookups_3

commit 66649ff891
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Jan 11 13:16:01 2014 +0200

    Some rewording in docs

commit 31b8faa627
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Dec 29 15:52:29 2013 +0200

    Cleanup based on review comments

commit 1016159f34
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Dec 28 18:37:04 2013 +0200

    Proof-of-concept fix for #16731

    Implemented only for SQLite and PostgreSQL, and only for startswith
    and istartswith lookups.

commit 193cd097ca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Dec 28 17:57:58 2013 +0200

    Fixed #11722 -- iexact=F() produced invalid SQL

commit 08ed3c3b49
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Dec 21 23:59:52 2013 +0200

    Made Lookup and Extract available from django.db.models

commit b99c8d83c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Dec 21 23:06:29 2013 +0200

    Fixed review notes by Loic

commit 049eebc070
Merge: ed8fab7 b80a835
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Dec 21 22:53:10 2013 +0200

    Merge branch 'master' into lookups_3

    Conflicts:
    	django/db/models/fields/__init__.py
    	django/db/models/sql/compiler.py
    	django/db/models/sql/query.py
    	tests/null_queries/tests.py

commit ed8fab7fe8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Dec 21 22:47:23 2013 +0200

    Made Extracts aware of full lookup path

commit 27a57b7aed
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Dec 1 21:10:11 2013 +0200

    Removed debugger import

commit 074e0f5aca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Dec 1 21:02:16 2013 +0200

    GIS lookup support added

commit 760e28e72b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Dec 1 20:04:31 2013 +0200

    Removed usage of Constraint, used Lookup instead

commit eac4776684
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Dec 1 02:22:30 2013 +0200

    Minor cleanup of Lookup API

commit 2adf50428d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sun Dec 1 02:14:19 2013 +0200

    Added documentation, polished implementation

commit 32c04357a8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Nov 30 23:10:15 2013 +0200

    Avoid OrderedDict creation on lookup aggregate check

commit 7c8b3a32cc
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Sat Nov 30 23:04:34 2013 +0200

    Implemented nested lookups

    But there is no support of using lookups outside filtering yet.

commit 4d219d4cde
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date:   Wed Nov 27 22:07:30 2013 +0200

    Initial implementation of custom lookups
2014-01-18 11:46:19 +02:00
Baptiste Mispelon f1b3ab9c21 Fixed #11629 -- Deprecated callable arguments to queryset methods.
Callable arguments were an untested and undocumented feature.
2013-12-18 18:40:00 +01:00
Denis Moskalets d4e578d0f6 Fixed #21552 -- Allowed the use of None for the iexact lookup.
Thanks Anubhav Joshi for the documentation.
2013-12-18 09:34:53 -05:00
Loic Bistuer 6685713869 Fixed E127 pep8 warnings. 2013-12-14 11:59:15 -05:00
Alex Hill ec73ce5d8a Fixed comment typos. 2013-11-30 07:59:32 -05:00
Christopher Medrela 7477a4ffde Fixed E125 pep8 warnings 2013-11-28 08:50:11 -05:00
Anssi Kääriäinen 52015b963d Used simpler queries for m2m clearing when possible.
Refs #21169
2013-11-27 19:45:05 +02:00
Anssi Kääriäinen ba6c9fae45 Removed Query.setup_joins() and join() argument outer_if_first.
Instead always create new joins as OUTER.
2013-11-07 12:57:02 +02:00
Anssi Kääriäinen 6fe2b001db Fixed #21376 -- New implementation for query join promotion logic
This commit introduced a new class JoinPromoter that can be used to
abstract away join promotion problems for complex filter conditions.
Query._add_q() and Query.combine() now use the new class.

Also, added a lot of comments about why join promotion is done the way
it is.

Thanks to Tim Graham for original report and testing the changes, and
for Loic Bistuer for review.
2013-11-07 12:53:26 +02:00
coagulant 3bc0d46a84 Fixed all E261 warnings 2013-11-02 18:20:39 -04:00
Anssi Kääriäinen bec0b2a8c6 Fixed #14511 -- bug in .exclude() query 2013-11-02 22:35:45 +02:00
Anssi Kääriäinen b44d42be6d Fixed #21366 -- regression in join promotion logic
The regression was caused by ecaba36028
and affected OR connected filters.
2013-11-02 20:44:19 +02:00
Alasdair Nicol b289fcf1bf Fixed #21288 -- Fixed E126 pep8 warnings 2013-10-21 08:31:30 -04:00
Marc Tamlyn 349c12d3f5 Fixed #16855 -- select_related() chains as expected.
select_related('foo').select_related('bar') is now equivalent to
select_related('foo', 'bar').

Also reworded docs to recommend select_related(*fields) over select_related()
2013-10-15 15:59:36 +01:00
Anssi Kääriäinen 93cc6dcdac Fixed #18414 -- qs.exists() for sliced distinct queries 2013-10-05 21:40:36 +03:00
Anssi Kääriäinen ecaba36028 Improved Query join promotion logic
There were multiple cases where join promotion was a bit too aggressive.
This resulted in using outer joins where not necessary.

Refs #21150.
2013-10-05 14:52:17 +03:00
Anssi Kääriäinen ff723d894d Fixed #20950 -- Instantiate OrderedDict() only when needed
The use of OrderedDict (even an empty one) was surprisingly slow. By
initializing OrderedDict only when needed it is possible to save
non-trivial amount of computing time (Model.save() is around 30% faster
for example).

This commit targetted sql.Query only, there are likely other places
which could use similar optimizations.
2013-09-14 20:52:17 +03:00
Simon Charette 11cd7388f7 Fixed #20989 -- Removed useless explicit list comprehensions. 2013-08-30 10:57:51 -04:00
Anssi Kääriäinen 13be3bfef1 Removed stale add_q() comment 2013-08-30 09:45:32 +03:00
Tim Graham c7d0ff0cad Fixed #20989 -- Removed explicit list comprehension inside dict() and tuple()
Thanks jeroen.pulles at redslider.net for the suggestion and
helper script.
2013-08-29 12:11:03 -04:00
Anssi Kääriäinen 7bc57a6d71 Fixed #11881 -- removed junk from aggregation subqueries
There were clauses that weren't needed in the subqueries. These were
ORDER BY, SELECT FOR UPDATE and related selections.
2013-08-19 16:15:09 +03:00
Anssi Kääriäinen 7737305a4f Fixed #12886 -- aggregation over sliced queryset 2013-08-19 16:00:17 +03:00
Anssi Kääriäinen dcdc579d16 Fixed #20874 -- bump_prefix() in nested subqueries
Also made some cleanup to build_filter() code by introducing submethods
solve_lookup_type() and prepare_lookup_value().
2013-08-13 14:11:52 +03:00
Anssi Kääriäinen c7739e30b2 Fixed #17424 -- annotate() + exclude() bug
The bug was already fixed by 01b9c3d519,
so only tests added.

At the same time promote_joins()'s uncoditional flag is gone, it isn't
needed for anything any more.
2013-08-07 12:53:33 +03:00
Curtis Maloney 07876cf02b Deprecated SortedDict (replaced with collections.OrderedDict)
Thanks Loic Bistuer for the review.
2013-08-04 07:09:39 -04:00
Florian Hahn e888a9b30d Fixed #15624 -- Made sure aggregations are present in SELECT 2013-07-31 16:27:58 +03:00
Anssi Kääriäinen c21e86ab9e Added field.attname to Options.name_map
The change also removed allow_explicit_fk from sql/query.py.
2013-07-31 16:02:36 +03:00
Anssi Kääriäinen 4bd5554721 Fixed #20782 -- qs.values().aggregate() failure
In the combination of .values().aggregate() the aggregate_select_mask
didn't include the aggregates added. This resulted in bogus query.

Thanks to Trac alias debanshuk for report.
2013-07-23 11:38:38 +03:00
Tim Graham 95eb68c98f Fixed #17339 -- Factor out has_result into database backend.
Thanks jonash.
2013-07-09 06:41:52 -04:00
Alex Gaynor 03d9566e0d A large number of stylistic cleanups across django/db/ 2013-07-08 10:39:54 +10:00
Florian Hahn 2f35c6f10f Fixed #14930 -- values_list() failure on qs ordered by extra column
Thanks lsaffre for the report and simon29, vicould, and Florian Hahn
for the patch.

Some changes done by committer.
2013-06-18 23:56:51 +03:00
Anssi Kääriäinen aa22cbd51a Fixed #20583 -- ORM always uses setup_joins() for join generation
There were a couple of places which used Query.join() directly. By
using setup_joins() in these places the code is more DRY, and in
addition there is no need to directly call field.get_joining_columns()
unless the field is the given join_field from get_path_info(). This
makes it easier to make sure a ForeignObject subclass generates joins
correctly in all cases.
2013-06-16 15:44:52 +03:00
Anssi Kääriäinen 89bf7a4525 Fixed #20528 -- regression in select_related join promotion
The join used by select_related was incorrectly INNER when the query
had an ORed filter for nullable join that was trimmed away. Fixed this
by forcing the join type to LOUTER even when a join was trimmed away
in ORed queries.
2013-06-14 16:29:06 +03:00
Anssi Kääriäinen 31fd64ad8a Fixed #20564 -- Generic relations exclude() regression
The patch for #19385 caused a regression in certain generic relations
.exclude() filters if a subquery was needed. The fix contains a
refactoring to how Query.split_exclude() and Query.trim_start()
interact.

Thanks to Trac alias nferrari for the report.
2013-06-06 01:54:46 +03:00
Aymeric Augustin 9c487b5974 Replaced an antiquated pattern.
Thanks Lennart Regebro for pointing it out.
2013-05-17 18:08:58 +02:00
Mike Fogel 74f3884ae0 Fixed #20413 - Respect Query.get_meta() 2013-05-15 12:55:30 -07:00
Anssi Kääriäinen e17fa9e877 Fixed #20091 -- Oracle null promotion for empty strings 2013-03-26 14:19:54 +02:00
Anssi Kääriäinen 97774429ae Fixed #19385 again, now with real code changes
The commit of 266de5f9ae included only
tests, this time also code changes included...
2013-03-24 18:40:40 +02:00
Anssi Kääriäinen 0ff12c28d0 Removed unused import 2013-03-21 00:50:48 +02:00
Anssi Kääriäinen f403653cf1 Fixed #19635 -- Made fields pickleable 2013-03-17 15:32:50 +02:00
Anssi Kääriäinen d3f00bd570 Refactored qs.add_q() and utils/tree.py
The sql/query.py add_q method did a lot of where/having tree hacking to
get complex queries to work correctly. The logic was refactored so that
it should be simpler to understand. The new logic should also produce
leaner WHERE conditions.

The changes cascade somewhat, as some other parts of Django (like
add_filter() and WhereNode) expect boolean trees in certain format or
they fail to work. So to fix the add_q() one must fix utils/tree.py,
some things in add_filter(), WhereNode and so on.

This commit also fixed add_filter to see negate clauses up the path.
A query like .exclude(Q(reversefk__in=a_list)) didn't work similarly to
.filter(~Q(reversefk__in=a_list)). The reason for this is that only
the immediate parent negate clauses were seen by add_filter, and thus a
tree like AND: (NOT AND: (AND: condition)) will not be handled
correctly, as there is one intermediary AND node in the tree. The
example tree is generated by .exclude(~Q(reversefk__in=a_list)).

Still, aggregation lost connectors in OR cases, and F() objects and
aggregates in same filter clause caused GROUP BY problems on some
databases.

Fixed #17600, fixed #13198, fixed #17025, fixed #17000, fixed #11293.
2013-03-13 10:44:49 +02:00
Anssi Kääriäinen d744c550d5 Fixed #19964 -- Removed relabel_aliases from some structs
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
2013-03-12 21:33:47 +02:00
Anssi Kääriäinen 06de130dae Fixed #12823 -- Was already fixed in master, tests added
Also added a little improvement to sql/query.py to get rid of
non-necessary IS NOT NULL check.
2013-02-28 15:05:48 +02:00
Anssi Kääriäinen 10f9ba046f Cleaned up join promotion in query.combine() with OR
Refs #19849
2013-02-20 21:45:36 +02:00
Anssi Kääriäinen edf93127bf Removed join() promote kwarg
The join promote=True was over-aggressive in select_related handling.
After that was removed, the only other user was query.combine(). That
use case is very easy to handle locally, so there is no more need for
the join(promote=True) flag.

Refs #19849.
2013-02-20 21:43:44 +02:00
Anssi Kääriäinen 3fef304ff2 Refactored negated IS NULL handling
This one cleaned up add_filter() negated filter generation. As a side
effect split_exclude() was cleaned up, too.

Refs #19849
2013-02-20 21:34:01 +02:00
Anssi Kääriäinen b4492a8ca4 Fixed #19837 -- Refactored split_exclude() join generation
The refactoring mainly concentrates on making sure the inner and outer
query agree about the split position. The split position is where the
multijoin happens, and thus the split position also determines the
columns used in the "WHERE col1 IN (SELECT col2 from ...)" condition.

This commit fixes a regression caused by #10790 and commit
69597e5bcc. The regression was caused
by wrong cols in the split position.
2013-02-19 01:10:49 +02:00
Anssi Kääriäinen 5cc0f5f8c1 Made Query.clear_ordering force_empty arg mandatory
Previously it was possible to call clear_ordering without the
force_empty argument. The result was that the query was still ordered
by model's meta ordering if that was defined. By making the arg
mandatory it will be easier to spot possible errors caused by assuming
clear_ordering will remove all ordering.

Thanks to Dylan Klomparens for the suggestion. Refs #19720.
2013-02-10 20:58:25 +02:00
Anssi Kääriäinen 23ca3a0194 Fixed #16759 -- Remove use of __deepcopy__ in qs.clone()
The original problem was that queryset cloning was really expensive
when filtering with F() clauses. The __deepcopy__ went too deep copying
_meta attributes of the models used. To fix this the use of
__deepcopy__ in qs cloning was removed.

This commit results in some speed improvements across the djangobench
benchmark suite. Most query_* tests are 20-30% faster, save() is 50%
faster and finally complex filtering situations can see 2x to order
of magnitude improvments.

Thanks to Suor, Alex and lrekucki for valuable feedback.
2013-01-08 19:17:13 +02:00
Anssi Kääriäinen a2396a4c8f Fixed #19173 -- Made EmptyQuerySet a marker class only
The guarantee that no queries will be made when accessing results is
done by new EmptyWhere class which is used for query.where and having.

Thanks to Simon Charette for reviewing and valuable suggestions.
2013-01-06 19:18:28 +02:00
Anssi Kääriäinen ce3c71faf1 Minor improvement to proxy model handling
Refs #19385
2012-12-30 11:20:27 +02:00
Anssi Kääriäinen 807eff7439 Made use of PathInfo.direct flag in trim_joins
Refs #19385
2012-12-30 11:20:13 +02:00
Anssi Kääriäinen 68985db482 Added Query.join_parent_model()
This simplifies especially compiler.py a lot, where almost the same
code was repeated multiple times.

Refs #19385
2012-12-30 11:19:35 +02:00
Anssi Kääriäinen 4511aeb6b8 Moved join path generation to Field
Refs #19385
2012-12-30 11:19:19 +02:00
Anssi Kääriäinen 9ef3cab40b Made sure join_field is always available in .join()
Refs #19385
2012-12-30 11:18:25 +02:00
Anssi Kääriäinen d407164c04 Fixed #18854 -- Join promotion in disjunction cases
The added promotion logic is based on promoting any joins used in only
some of the childs of an OR clause unless the join existed before the
OR clause addition.
2012-12-20 21:45:15 +02:00
Anssi Kääriäinen 3dcd435a0e Fixed #19500 -- Solved a regression in join reuse
The ORM didn't reuse joins for direct foreign key traversals when using
chained filters. For example:
    qs.filter(fk__somefield=1).filter(fk__somefield=2))
produced two joins.

As a bonus, reverse onetoone filters can now reuse joins correctly

The regression was caused by the join() method refactor in commit
68847135bc

Thanks for Simon Charette for spotting some issues with the first draft
of the patch.
2012-12-20 21:27:00 +02:00
Anssi Kääriäinen c64b57d166 Replaced '__' with LOOKUP_SEP in sql/query.py
Thanks to Simon Charette for report.
2012-12-17 21:15:01 +02:00
Anssi Kääriäinen 69597e5bcc Fixed #10790 -- Refactored sql.Query.setup_joins()
This is a rather large refactoring. The "lookup traversal" code was
splitted out from the setup_joins. There is now names_to_path() method
which does the lookup traveling, the actual work of setup_joins() is
calling names_to_path() and then adding the joins found into the query.

As a side effect it was possible to remove the "process_extra"
functionality used by genric relations. This never worked for left
joins. Now the extra restriction is appended directly to the join
condition instead of the where clause.

To generate the extra condition we need to have the join field
available in the compiler. This has the side-effect that we need more
ugly code in Query.__getstate__ and __setstate__ as Field objects
aren't pickleable.

The join trimming code got a big change - now we trim all direct joins
and never trim reverse joins. This also fixes the problem in #10790
which was join trimming in null filter cases.
2012-12-16 17:23:26 +02:00
Anssi Kääriäinen f811649710 Fixed #18816 -- Removed "trim" argument from add_filter()
The trim argument was used by split_exclude() only to trim the last
join from the given lookup. It is cleaner to just trim the last part
from the lookup in split_exclude() directly so that there is no need
to burden add_filter() with the logic needed for only split_exclude().
2012-12-16 16:56:28 +02:00
Tai Lee 6ebf115206 Fixed #14694 -- Made ``defer()`` work with reverse relations
Reverse o2o fields are now usable with defer.
2012-11-28 18:17:10 +02:00
Anssi Kääriäinen 08b4a22293 Updated stale docstring of setup_joins 2012-11-24 16:26:05 +02:00
Anssi Kääriäinen 90b86291d0 Fixed #18375 -- Removed dict-ordering dependency for F-expressions
F() expressions reuse joins like any lookup in a .filter() call -
reuse multijoins generated in the same .filter() call else generate
new joins. Also, lookups can now reuse joins generated by F().

This change is backwards incompatible, but it is required to prevent
dict randomization from generating different queries depending on
.filter() kwarg ordering. The new way is also more consistent in how
joins are reused.
2012-11-23 19:53:04 +02:00
Anssi Kääriäinen ce1af8d702 Removed use of SortedDict for query.alias_refcount
This will have a smallish impact on performance. Refs #19276.
2012-11-13 22:48:25 +02:00
Anssi Kääriäinen 68847135bc Removed dupe_avoidance from sql/query and sql/compiler.py
The dupe avoidance logic was removed as it doesn't seem to do anything,
it is complicated, and it has nearly zero documentation.

The removal of dupe_avoidance allowed for refactoring of both the
implementation and signature of Query.join(). This refactoring cascades
again to some other parts. The most significant of them is the changes
in qs.combine(), and compiler.select_related_descent().
2012-10-31 08:19:44 +02:00
Anssi Kääriäinen 11699ac4b5 Fixed #19190 -- Refactored Query select clause attributes
The Query.select and Query.select_fields were collapsed into one list
because the attributes had to be always in sync. Now that they are in
one attribute it is impossible to edit them out of sync.

Similar collapse was done for Query.related_select_cols and
Query.related_select_fields.
2012-10-27 02:13:02 +03:00
Anssi Kääriäinen bd6d9ea87c Fixed regression caused by #19102 2012-10-25 19:13:10 +03:00
Anssi Kääriäinen f64a5ef404 Fixed #19102 -- Fixed fast-path delete for modified SELECT clause cases
There was a bug introduced in #18676 which caused fast-path deletes
implemented as "DELETE WHERE pk IN <subquery>" to fail if the SELECT
clause contained additional stuff (for example extra() and annotate()).

Thanks to Trac alias pressureman for spotting this regression.
2012-10-25 17:16:44 +03:00
Anssi Kääriäinen a62d53c032 Fixed #19087 -- Ensured query's base table is never LOUTER joined
This fixes a regression created by join promotion logic refactoring:
01b9c3d519

Thanks to Ivan Virabyan for the report.
2012-10-08 18:40:09 +03:00
Malcolm Tredinnick c4aa26a983 Internal refactoring; moving LOOKUP_SEP up one level.
In an ideal world, nothing except django.db.models.query should have to
import stuff from django.models.sql.*. A few things were needing to get
hold of sql.constants.LOOKUP_SEP, so this commit moves it up to
django.db.models.constants.LOOKUP_SEP.

There are still a couple of places (admin) poking into sql.* to get
QUERY_TERMS, which is unfortunate, but a slightly different issue and
harder to adjust.
2012-09-08 19:51:36 -04:00
Alex Gaynor 0e296131bb Cleaned up some small bits of the ORM, including removing an import *. 2012-09-07 10:58:17 -04:00
Anssi Kääriäinen 01b9c3d519 Fixed #16715 -- Fixed join promotion logic for nested nullable FKs
The joins for nested nullable foreign keys were often created as INNER
when they should have been OUTER joins. The reason was that only the
first join in the chain was promoted correctly. There were also issues
with select_related etc.

The basic structure for this problem was:
  A -[nullable]-> B -[nonnull]-> C

And the basic problem was that the A->B join was correctly LOUTER,
the B->C join not.

The major change taken in this patch is that now if we promote a join
A->B, we will automatically promote joins B->X for all X in the query.
Also, we now make sure there aren't ever join chains like:
   a LOUTER b INNER c
If the a -> b needs to be LOUTER, then the INNER at the end of the
chain will cancel the LOUTER join and we have a broken query.

Sebastian reported this problem and did also major portions of the
patch.
2012-08-25 14:14:45 +03:00
Anssi Kääriäinen a193372753 Fixed #17886 -- Fixed join promotion in ORed nullable queries
The ORM generated a query with INNER JOIN instead of LEFT OUTER JOIN
in a somewhat complicated case. The main issue was that there was a
chain of nullable FK -> non-nullble FK, and the join promotion logic
didn't see the need to promote the non-nullable FK even if the
previous nullable FK was already promoted to LOUTER JOIN. This resulted
in a query like a LOUTER b INNER c, which incorrectly prunes results.
2012-08-21 21:23:57 +03:00
Anssi Kääriäinen c1684e3dcb Fixed #18731 -- Cleaned up split_exclude's use of can_reuse
The outer query's set of reusable joins (can_reuse) was passed to the
inner query's add_filter call. This was incorrect.
2012-08-12 21:40:22 +03:00
Alex Gaynor 4c97101b1f remove a bunch of unnescesarry iterkeys() calls 2012-08-08 07:33:15 -07:00
Aymeric Augustin c5ef65bcf3 [py3] Ported django.utils.encoding.
* Renamed smart_unicode to smart_text (but kept the old name under
  Python 2 for backwards compatibility).
* Renamed smart_str to smart_bytes.
* Re-introduced smart_str as an alias for smart_text under Python 3
  and smart_bytes under Python 2 (which is backwards compatible).
  Thus smart_str always returns a str objects.
* Used the new smart_str in a few places where both Python 2 and 3
  want a str.
2012-08-07 12:00:22 +02:00
Aymeric Augustin ee191715ea [py3] Fixed access to dict keys/values/items. 2012-08-07 12:00:22 +02:00
Alex Gaynor 52df0d50b0 Switched to use a more idiomatic construct. 2012-07-17 07:01:01 -07:00
Anssi Kääriäinen fcad6c48f0 Fixed #17497 -- Corrected FieldError message in add_fields()
The erroneous message was user visible in values_list() calls.

Thanks to ojii for report and review, and to antoviaque for the patch.
2012-07-17 12:49:46 +03:00
Anssi Kääriäinen b6c356b7bb Fixed #17485 -- Made defer work with select_related
This commit tackles a couple of issues. First, in certain cases there
were some mixups if field.attname or field.name should be deferred.
Field.attname is now always used.

Another issue tackled is a case where field is both deferred by
.only(), and selected by select_related. This case is now an error.

A lot of thanks to koniiiik (Michal Petrucha) for the patch, and
to Andrei Antoukh for review.
2012-06-26 18:08:42 +03:00
Claude Paroz 169b1a404c Replaced foo.next() by next(foo).
This new syntax for next() has been introduced in Python 2.6 and is
compatible with Python 3.
2012-05-10 20:15:49 +02:00
Anssi Kääriäinen c2e1ecb4b1 Fix proxy model Query.remove_inherited_models()
Fixed #18248 -- proxy models were added to included_inherited_models
in sql.query.Query. The variable is meant to be used for multitable
inheritance only. This mistake caused problems in situations where
proxy model's query was reused.
2012-05-09 20:33:31 +03:00
Adrian Holovaty 6ff118cdb9 Fixed #17644 -- Changed Query.alias_map to use namedtuples
This makes the code easier to understand and may even have a benefit in memory usage (namedtuples instead of dicts). Thanks, lrekucki and akaariai
2012-04-29 17:50:48 -05:00
Claude Paroz 3904b74a3f Fixed #18013 -- Use the new 'as' syntax for exceptions.
Thanks Clueless for the initial patch.
Note that unittest has been purposely left out (external package only used by Python 2.6).
2012-04-29 20:57:15 +02:00
Anssi Kääriäinen 76c3314b6d Removed unused variable from sql/query.py 2012-04-29 21:23:38 +03:00
Anssi Kääriäinen 584e2c0337 Prevent Oracle from changing field.null to True
Fixed #17957 -- when using Oracle and character fields, the fields
were set null = True to ease the handling of empty strings. This
caused problems when using multiple databases from different vendors,
or when the character field happened to be also a primary key.

The handling was changed so that NOT NULL is not emitted on Oracle
even if field.null = False, and field.null is not touched otherwise.

Thanks to bhuztez for the report, ramiro for triaging & comments,
ikelly for the patch and alex for reviewing.
2012-04-29 19:25:46 +03:00
Anssi Kääriäinen 5aa51fa999 Simplified QuerySet field.null handling
QuerySet had previously some complex logic for dealing with nullable
fields in negated add_filter() calls. It seems the logic is leftover
from a time where the WhereNode wasn't as intelligent in handling
field__in=[] conditions.

Thanks to aaugustin for comments on the patch.
2012-04-29 16:27:25 +03:00
Anssi Kääriäinen 4f62352412 Fixed #18014 -- Removed rev_join_map from sql/query.py.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17878 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-04-08 12:29:05 +00:00
Aymeric Augustin ce7113ccbe Used SortedDict instead of dict to avoid random errors that may occur when dict randomization is enabled in Python. Refs #17758. Thanks Łukasz Rekucki.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17777 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-03-22 18:09:22 +00:00
Carl Meyer 354c84d277 Fixed #17678 -- Corrected setup of _meta.proxy_for_model and added _meta.concrete_model. Thanks Anssi Kääriäinen.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17573 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-02-22 05:26:50 +00:00
Julien Phalip d02ba7f4ee Fixed #11670 -- Prevented genuine model fields named 'year', 'month', 'gt', 'lt' etc. from being mistaken for lookup types in lookups across relations. Thanks to andy for the report, to jpwatts for the initial patch and to Anssi Kääriäinen and Alex Gaynor for the reviews.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17450 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-02-05 07:11:53 +00:00
Ramiro Morales 287565779d Added support for modifying the effect of ``DISTINCT`` clauses so they
only consider some fields (PostgreSQL only).

For this, the ``distinct()`` QuerySet method now accepts an optional
list of model fields names and generates ``DISTINCT ON`` clauses on
these cases. Thanks Jeffrey Gelens and Anssi Kääriäinen for their work.

Fixes #6422.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@17244 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-12-22 20:42:40 +00:00
Adrian Holovaty 20c8aa2a20 Fixed various dodgy behaviours
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17226 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-12-17 17:37:24 +00:00
Karen Tracey b8353016b7 Fixed #13640: Avoid generating an exception when a model has an attribute named 'evaluate'. Thanks LukaszKorzybski and tobias.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17093 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-11-13 15:09:08 +00:00
Alex Gaynor 345c4c4629 Remove a bunch of deadcode/dead imports.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16794 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-09-10 22:31:38 +00:00
Alex Gaynor 7deb25b8dd Fixed #7596. Added Model.objects.bulk_create, and make use of it in several places. This provides a performance benefit when inserting multiple objects. THanks to Russ for the review, and Simon Meers for the MySQl implementation.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16739 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-09-09 19:22:28 +00:00
Malcolm Tredinnick 7182cd2284 Allow "pk" as a field alias in QuerySet.only() calls.
Thanks to GDorn for the patch. Fixed #15494.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@16668 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-23 06:29:01 +00:00
Malcolm Tredinnick 2e56066a5b Fixed an isnull=False filtering edge-case. Fixes #15316.
The bulk of this patch is due to some fine analysis from Aleksandra
Sendecka.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@16656 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-23 03:38:42 +00:00
Malcolm Tredinnick c3a0dcf6e9 Added convenience method for viewing Query SQL without params.
This is the old Query.as_sql() method revived: it's like Query.__str__,
but the parameters aren't substituted into the placeholders. Thus, it's
a more accurate representation of the SQL the (default) backend will
see. Entirely internal.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@16655 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-23 03:38:28 +00:00
Russell Keith-Magee 3afe409d0e Fixed #14876 -- Ensure that join promotion works correctly when there are nullable related fields. Thanks to simonpercivall for the report, oinopion and Aleksandra Sendecka for the original patch, and to Malcolm for helping me wrestle the edge cases to the ground.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16648 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-22 07:40:12 +00:00
Jannis Leidel dbffffa7dc Fixed #5535 -- Allow using an explicit foreign key in get() calls. Thanks, Michal Petrucha.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16473 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-06-28 10:16:18 +00:00
Ramiro Morales 15793309e1 Fixed #14476 -- Fixed resolution of automatically generated annotation names so e.g. filtering based on them works. Thanks dirleyls for the report and patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16252 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-05-21 13:12:23 +00:00
Ramiro Morales 07bfc76ecf Fixed #15790 -- Fixed QuerySet only() and defer() methods behavior with proxy models. Thanks Michal Modzelewzki for the report and patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16228 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-05-15 00:56:01 +00:00
Luke Plant db5807bdb1 Fixed #15823 - incorrect join condition when combining Q objects
Thanks to dcwatson for the excellent report and patch.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@16159 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-05-05 20:26:26 +00:00
Jannis Leidel c77372cad0 Fixed #14729 -- RawQuerySet.__repr__ fails when params passed as list. Thanks, intgr for ticket and accuser for patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16088 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-04-22 18:17:26 +00:00
Jacob Kaplan-Moss 8f0f73c7b8 Fixed #2705: added a `select_for_update()` clause to querysets.
A number of people worked on this patch over the years -- Hawkeye, Colin Grady,
KBS, sakyamuni, anih, jdemoor, and Issak Kelly. Thanks to them all, and
apologies if I missed anyone.

Special thanks to Dan Fairs for picking it up again at the end and seeing this
through to commit.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@16058 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-04-20 20:42:07 +00:00
Adrian Holovaty 3b6c5e5eb2 Made some negligible docstring fixes while I was poking around in the depths of query.py
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16025 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-04-11 21:48:47 +00:00
Adrian Holovaty 13864703bc Removed a bunch more Python 2.4 workarounds now that we don't support that version. Refs #15702 -- thanks to jonash for the patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15927 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-03-28 02:11:19 +00:00
Jacob Kaplan-Moss fd2f18008c Fixed #14733: no longer "validate" .raw() queries.
Turns out that a lot more than just SELECT can return data, and this list is
very hard to define up front in a cross-database manner. So let's just assume
that anyone using raw() is at least halfway competant and can deal with
the error messages if they don't use a data-returning query.

Thanks to Christophe Pettus for the patch.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@15803 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-03-14 19:49:53 +00:00
Russell Keith-Magee b7c41c1fbb Fixed #12252 -- Ensure that queryset unions are commutative. Thanks to benreynwar for the report, and draft patch, and to Karen and Ramiro for the review eyeballs and patch updates.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15726 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-03-03 13:51:54 +00:00
Russell Keith-Magee d3b38d578f Fixed #13815 -- Ensure that reverse exclude lookups on nullable foreign keys exclude null values. Thanks to bpeschier for the report and patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15458 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-02-08 14:06:02 +00:00
Carl Meyer 84291b7b84 Fixed #15161 - Corrected handling of ManyToManyField with through table using to_field on its ForeignKeys. Thanks to adehnert for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15330 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-01-26 19:10:08 +00:00
Carl Meyer a9b4f2a400 Tweaked r15303 to avoid O(N) lookup of field name. Thanks to Alex's eagle eyes.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15324 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-01-26 08:39:30 +00:00
Carl Meyer 227c5e80db Fixed #11319 - Added lookup support for ForeignKey.to_field. Also reverted no-longer-needed model formsets workaround for lack of such support from r10756. Thanks Russell and Alex for review.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15303 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-01-25 03:14:28 +00:00
Alex Gaynor 92d4352283 Cleaned up the ORM slightly.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15203 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-01-14 05:04:14 +00:00
Alex Gaynor 29de7ee9cb Fixed #11293 -- fixed using Q objects to generate ORs with aggregates.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@15173 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-01-11 01:00:50 +00:00
Alex Gaynor 678f626c24 Fixed a suite of errors in the ORM -- a) fixed calling values_list().values_list() and changing whether the results are flat, b) fixed an issue with fields on the left-hand side of what becomes the HAVING clause not being included in the GROUP BY clause, and c) fixed a bug with fields from values() calls not being included in the GROUP BY clause. This fixed the recent test failures under postgresql.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@14715 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-11-26 14:24:08 +00:00
Alex Gaynor 0eb31d3dc9 Fixed #14423 -- corrected incorrect SQL being generated when a nullable, inherited field was used in an exclude. Thanks to PhiR_42 for the patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@14600 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-11-17 16:46:50 +00:00
Alex Gaynor 4276b5197b Fixed #12687 -- fixed an issue with aggregates and counts in conjunction with annotations where the QuerySet was provably empty.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@14586 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-11-17 02:57:14 +00:00
Russell Keith-Magee c4e766c100 Fixed #13882 -- Removed an unnecessary nested where clause introduced on __isnull queries. Thanks to Alex Gaynor for the report and patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@13456 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-07-30 04:06:27 +00:00
Russell Keith-Magee 8b1aebbf35 Fixed #12851 -- Another attempt at fixing select_related() with inherited models, this time with only(). Thanks to phxx for the test case.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@13059 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-04-30 16:32:48 +00:00
Russell Keith-Magee 4e97d7f8e4 Fixed #12851 -- Corrected the loading of values when select_related() is used on inherited models. Thanks to phxx for the report and test case.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@13054 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-04-30 14:07:47 +00:00
Russell Keith-Magee 442ee687df Fixed #13349 -- Ensure that raw queries evaluate the entire query if the backend doesn't support chunked reads. Thanks to Alex Gaynor for the report and patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12978 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-04-15 13:01:51 +00:00
Russell Keith-Magee 8f8743a69e Fixed #12429 -- Modified RawQuery to provide some facilities required by Oracle. Thanks to Karen Tracey for the testing help.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12907 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-04-02 14:44:16 +00:00
Russell Keith-Magee f7cf58ac0e Fixed #12429 -- Ensure that raw queries call resolve_columns if the backend defines it. This ensures (as much as possible) that the model values returned by a raw query match that in normal queries. Thanks to Ian Kelly for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12904 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-04-01 16:48:16 +00:00
Karen Tracey 9df71371c2 Fixed #12822: Don't copy the _aggregate_select_cache when cloning a query set,
as that can lead to incorrect SQL being generated for the query. Thanks to mat
for the report and test, tobias for the fix, and Alex for review.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@12830 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-03-21 20:47:52 +00:00
Russell Keith-Magee 9f6a82fafb Fixed #13156 -- Ensure that exists() queries are as fast as they can be. Thanks to Jerome Leclanche for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12810 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-03-20 10:37:57 +00:00
Russell Keith-Magee 3508a86ddf Fixed #12876 -- Corrected a problem with recursive relations under deepcopy. Thanks to elachuni for the patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12700 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-03-07 07:11:22 +00:00
Justin Bronn c4699b0b8a Fixed #12806 -- Added an implementation of `RawQuerySet.__getitem__`. Thanks, Bruno Renié.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12504 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-02-23 05:22:12 +00:00
Justin Bronn 6ed7bd5609 Fixed #12855 -- QuerySets with `extra` where parameters now combine correctly. Thanks, Alex Gaynor.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12502 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-02-23 04:39:39 +00:00
Karen Tracey bb6b9aa472 Fixed #11535: Corrected SQL generated for queries involving generic relations and ORed Q objects. Thanks to brianglass for report, tobias for fix and test, and Alex for review.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@12405 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2010-02-10 19:21:52 +00:00
Russell Keith-Magee efbace6b7e Fixed #12433 -- Corrected a typo in the text of the exception raised by RawQuery. Thanks to bjunix for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11973 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-12-24 05:35:43 +00:00
Ian Kelly cdf5ad4217 Changes to get raw queries working on the oracle backend.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11968 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-12-22 21:05:15 +00:00
Russell Keith-Magee ff60c5f9de Fixed #1142 -- Added multiple database support.
This monster of a patch is the result of Alex Gaynor's 2009 Google Summer of Code project.
Congratulations to Alex for a job well done.

Big thanks also go to:
 * Justin Bronn for keeping GIS in line with the changes,
 * Karen Tracey and Jani Tiainen for their help testing Oracle support
 * Brett Hoerner, Jon Loyens, and Craig Kimmerer for their feedback.
 * Malcolm Treddinick for his guidance during the GSoC submission process.
 * Simon Willison for driving the original design process
 * Cal Henderson for complaining about ponies he wanted.

... and everyone else too numerous to mention that helped to bring this feature into fruition.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@11952 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-12-22 15:18:51 +00:00
Jacob Kaplan-Moss 20ad30713e Fixed #11863: added a `Model.objects.raw()` method for executing raw SQL queries and yield models.
See `docs/topics/db/raw.txt` for details.

Thanks to seanoc for getting the ball rolling, and to Russ for wrapping things up.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@11921 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-12-20 02:46:58 +00:00
Jacob Kaplan-Moss 574eafe4c0 Fixed #11753 - Q objects with callables no longer explode on Python 2.4. Thanks, Jeremy Dunck.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11901 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-12-17 22:06:41 +00:00
Karen Tracey c54b8ec2f5 Fixed #12079: Changed has_results to get a single result, thus preventing exists() from always returning True on backends that support chunked reads.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11654 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-10-25 16:32:07 +00:00
Jacob Kaplan-Moss b79702b2de Fixed #11402: added a `QuerySet.exists()` method. Thanks, Alex Gaynor.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11646 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-10-24 00:28:39 +00:00
Russell Keith-Magee 151d88af4e Fixed #11082 -- Ensured that subqueries used in an exclude(X__in=) clause aren't pre-evaluated. Thanks to Henry Andrews for the report, and clement for the fix.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10929 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-06-06 13:35:33 +00:00
Russell Keith-Magee 512ee0f528 Fixed #10572 -- Corrected the operation of the defer() and only() clauses when used on inherited models.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10926 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-06-06 06:14:05 +00:00
Gary Wilson Jr bd58a3972b Fixed #11216 and #11218 -- Corrected a few typos, thanks buriy.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10861 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-05-29 04:35:10 +00:00
Jacob Kaplan-Moss baaf29895c Fixed #10953, #10955: proxies of proxies now work correctly, though I still don't quite understand why you'd want to do such a thing. Thanks, Armin Ronacher.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10738 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-05-11 10:10:03 +00:00
Russell Keith-Magee 5e2d38465a Fixed #10847 -- Modified handling of extra() to use a masking strategy, rather than last-minute trimming. Thanks to Tai Lee for the report, and Alex Gaynor for his work on the patch.
This enables querysets with an extra clause to be used in an __in filter; as a side effect, it also means that as_sql() now returns the correct result for any query with an extra clause.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@10648 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-30 15:40:09 +00:00
Russell Keith-Magee 4a4d3e2133 Fixed #10796 -- Corrected a pickling problem introduced by [10522]. Thanks to carljm for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10526 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-11 15:25:15 +00:00
Russell Keith-Magee e12e0e18a4 Fixed #10197 -- Corrected pickling of querysets when a subset of fields was selected.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10522 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-11 13:20:51 +00:00
Russell Keith-Magee 0fff47c90f Fixed #10766 -- Raise an error when annotate() references another aggreagte(). Thanks to aseering@mit.edu for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10521 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-11 12:09:34 +00:00
Russell Keith-Magee 98ef7e85bd Fixed #10666 -- Corrected the handling of inherited fields with aggregate() and annotate(). Thanks to julienb for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10446 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-08 13:19:48 +00:00
Malcolm Tredinnick 2b1934ff3c Fixed a problem when computing deferred fields on multiple related models.
Fixed #10710, as this fixes the second bug reported there.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@10384 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-04 05:35:01 +00:00
Jacob Kaplan-Moss 25130b9675 Fixed #10670: fixed reusing QuerySets previously used in a filter expression. Thanks, Alex Gaynor.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10357 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-02 01:49:12 +00:00
Justin Bronn 03de1fe5f4 Fixed #9871 -- Geometry objects are now returned in dictionaries and tuples returned by `values()` and `values_list()`, respectively; updated `GeoQuery` methods to be compatible with `defer()` and `only`; removed defunct `GeomSQL` class; and removed redundant logic from `Query.get_default_columns`.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10326 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-04-01 16:01:50 +00:00
Malcolm Tredinnick fd46f673bd Fixed #9926 -- Fixes for some select_related() situations.
Using select_related(...) across a nullable relation to a multi-table
model inheritance situation no longer excludes results. Thanks to AdamG
for a test demonstrating part of the problem.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@10136 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-24 03:59:38 +00:00
Malcolm Tredinnick 29050ef999 Fixed #5420 -- Added support for delayed loading of model fields.
In extreme cases, some fields are expensive to load from the database
(e.g. GIS fields requiring conversion, or large text fields). This
commit adds defer() and only() methods to querysets that allow the
caller to specify which fields should not be loaded unless they are
accessed.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@10090 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-19 09:06:04 +00:00
Malcolm Tredinnick 61a2708c41 Fixed #10356 -- Added pure-Python inheritance for models (a.k.a proxy models).
Large portions of this are needed for #5420, so I implemented it fully.
Thanks to Ryan Kelly for an initial patch to get this started.

Refs #5420.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@10083 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-18 09:47:08 +00:00
Russell Keith-Magee 84ce18fc9b Fixed #10425 -- Corrected the interaction of .count() with .annotate() when .values() is also involved. Thanks to kmassey for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10053 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-14 04:32:52 +00:00
Ian Kelly f725658299 Fixed a case where column aliases weren't being quoted properly.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10048 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-13 17:20:52 +00:00
Malcolm Tredinnick 923f78f504 Improved table join handling for comparisons against NULL.
This fixes a broad class of bugs involving filters that look for missing
related models and fields. Most of them don't seem to have been reported
(the added tests cover the root cause). The exception is that this has
also fixed #9868.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9979 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-06 02:02:09 +00:00
Malcolm Tredinnick dfd7a6c781 Fixed #10251 -- Fixed model inheritance when there's also an explicit pk field.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9970 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-04 07:20:08 +00:00
Malcolm Tredinnick 0e93f60c7f Fixed #10362 -- An update() that only affects a parent model no longer crashes.
This includes a fairly large refactor of the update() query path (and
the initial portions of constructing the SQL for any query). The
previous code appears to have been only working more or less by accident
and was very fragile.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9967 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-04 05:34:01 +00:00
Malcolm Tredinnick c6a404d1e9 Fixed insert/update handling when no database interaction is required.
Fixed #10205 as part of this.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9926 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-01 01:56:59 +00:00
Malcolm Tredinnick bbea457fbb Fixed #10202 -- Fixed another couple of slicing edge-cases with querysets.
Patch from Alex Gaynor and Ramiro Morales.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9924 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-03-01 01:24:03 +00:00
Malcolm Tredinnick 415ffa8df5 Fixed #10028 -- Fixed a problem when ordering by related models.
Some results were inadvertently being excluded if we were ordering across a
nullable relation which itself ordering by a non-nullable relation.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9916 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-02-28 02:59:40 +00:00
Ian Kelly 8ffe8981f6 Fixed #10290: do not use aliases when adding extra_selects to the GROUP BY clause, to generate compliant sql that will be accepted by Oracle.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9905 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-02-25 23:56:00 +00:00
Russell Keith-Magee 542709d0d1 Fixed #10182 -- Corrected realiasing and the process of evaluating values() for queries with aggregate clauses. This means that aggregate queries can now be used as subqueries (such as in an __in clause). Thanks to omat for the report.
This involves a slight change to the interaction of annotate() and values() clauses that specify a list of columns. See the docs for details.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9888 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-02-23 14:47:59 +00:00
Russell Keith-Magee 2b1bb716ff Fixed #10248 -- Corrected handling of the GROUP BY clause when using a DateQuerySet. Thanks to Alex Gaynor for the report and patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9839 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-02-16 12:38:11 +00:00
Russell Keith-Magee fb64ea7896 Fixed #10132 -- Corrected the interaction of extra() queries with the values() clause. Thanks to Glen Maynard for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9838 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-02-16 12:29:31 +00:00
Russell Keith-Magee 8f410a577b Fixed #10127 -- Corrected handling of select_related() in annotate() calls. Thanks to Sylvain Pasche <sylvain.pasche@gmail.com> for the report and test case.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9805 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-02-02 13:43:18 +00:00
Russell Keith-Magee cf37e4624a Fixed #7210 -- Added F() expressions to query language. See the documentation for details on usage.
Many thanks to:
    * Nicolas Lara, who worked on this feature during the 2008 Google Summer of Code.
    * Alex Gaynor for his help debugging and fixing a number of issues.
    * Malcolm Tredinnick for his invaluable review notes.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9792 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-29 10:46:36 +00:00
Russell Keith-Magee 08dd4176ed Refs #10113 -- Modified the generated SQL to remove redundant GROUP BY elements, and modified the test added in [9788] to remove a test result that depended on a potentially ambiguous database ordering.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9791 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-24 14:16:25 +00:00
Russell Keith-Magee 15b1675af9 Fixed #10113 -- Ensured that joined fields mentioned in order_by() are included in the GROUP_BY clause on those backends that require it. Thanks to Koen Biermans <koen.biermans@werk.belgie.be> for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9788 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-24 06:53:54 +00:00
Russell Keith-Magee 0e15932be3 Fixed #10089 -- Corrected handling of aggregates when the query set contains no items (and the cursor returns None). Thanks to Kyle Fox for the report, and david for the initial patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9786 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-23 11:03:48 +00:00
Russell Keith-Magee ed3d2735a2 Fixed #10100 -- Corrected handling of the negation required exclude() clauses that reference aggregate columns. Thanks to Anossov for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9785 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-23 07:07:51 +00:00
Russell Keith-Magee 39516803f1 Fixed #10064 -- Corrected handling of aggregate queries that also use select_related(). Thanks to olivius for the report.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9781 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-20 12:23:48 +00:00
Russell Keith-Magee cc4e4d9aee Fixed #3566 -- Added support for aggregation to the ORM. See the documentation for details on usage.
Many thanks to:
 * Nicolas Lara, who worked on this feature during the 2008 Google Summer of Code.
 * Alex Gaynor for his help debugging and fixing a number of issues.
 * Justin Bronn for his help integrating with contrib.gis.
 * Karen Tracey for her help with cross-platform testing.
 * Ian Kelly for his help testing and fixing Oracle support.
 * Malcolm Tredinnick for his invaluable review notes.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9742 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-15 11:06:34 +00:00
Malcolm Tredinnick c5bdfab9ae Fixed #9985 -- qs.values_list(...).values(...) was constructing incorrect SQL.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9717 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-08 05:49:03 +00:00
Malcolm Tredinnick d00c013cc8 Remove any ordering from a nested select query (in the where-clause).
This fixes r9701 for Oracle (the test introduced there did its job and failed
previously).

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9714 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-08 05:15:41 +00:00
Malcolm Tredinnick f747b61c20 Nested query support.
This extends previous functionality that allowed passing Query objects as the
rvals to filters. You can now pass QuerySets, which requires less poking at
opaque attributes. See the documentation of the "__in" lookup type for the
details.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9701 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-05 11:47:48 +00:00
Malcolm Tredinnick 062a94ef45 Reconciling where- and having-clause behaviour.
Extricated the code that works directly with SQL columns (standard
"where" stuff) from the the code that takes SQL fragments and combines
it with lookup types and values. The latter portion is now more
generally reusable. Any existing code that was poking at Query.having
will now break in very visible ways (no subtle miscalculations, which is
a good thing).

This patch, en passant, removes the existing "having" test, since the
new implementation requires more setting up than previously. The
aggregates support (currently in a separate codebase) has tests for this
functionality that work as a replacement for the removed test.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@9700 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2009-01-05 11:47:14 +00:00
Malcolm Tredinnick a1cbeb9afb If an SQL query doesn't specify any ordering, avoid the implicit sort
that happens with MySQL when a "GROUP BY" clause is included. This is a
backend-specific operation, so any other databases requiring similar
encouragement can have a function added to their own backend code.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9637 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-12-10 05:19:27 +00:00
Malcolm Tredinnick 5570b0766d Fixed #9778 -- Added some special casing of the "Join on field 'abc'" error
message. It now gives an extra hint if there's a chance you just made a typo in
the lookup type.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9620 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-12-09 07:03:52 +00:00
Malcolm Tredinnick 33c0f0de67 This fixes a group of problems in the SQL created by QuerySet.exclude() when
used in a few situations where NULL results can appear.

Fixed #8921 (the only ticket I know of that noticed any of these).


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9590 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-12-08 02:39:51 +00:00
Malcolm Tredinnick d4f0ae42a2 Fixed #9188 -- Fixed a case where we were generating syntactically invalid SQL in some exclude() queries.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9588 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-12-07 05:48:01 +00:00
Malcolm Tredinnick 3dfbaae32b Fixed #9307 -- Added the ability to pickle the Query class used by the Oracle
backend.

This allows Querysets to be cached for Oracle and should provide a model for
adding pickling support to other (external) database backends that need a
custom Query class.

Thanks to Justin Bronn for some assistance with this patch.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9272 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-10-24 09:36:22 +00:00
Malcolm Tredinnick 9319dc496c Fixed #9406 -- Ensure that each database column is only represented once in the
"ORDER BY" clause of an SQL statement.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9251 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-10-24 06:09:47 +00:00
Malcolm Tredinnick 559aca7d78 A queryset that has had ordering removed (order_by()) can have ordering added
again later (order_by('foo')). Or, at least, it can now. Thanks to Ilya
Novoselov for diagnosing the problem here.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9206 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-10-08 09:45:27 +00:00
Malcolm Tredinnick 268ef594ac Applying a limit to a queryset that already had an upper limit of 0 wasn't
working properly.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9201 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-10-08 08:37:35 +00:00
Malcolm Tredinnick 922aba3def Fixed a missing variable initialisation deep in the Query bowels.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9081 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-22 00:31:22 +00:00
Malcolm Tredinnick e47cc781d8 A bug from queryset-refactor days: although the Query class has "group_by" and
"having" attributes, only the former was included in the resulting SQL, meaning
subclasses had to completely duplicate Query.as_sql() if they were using any
kind of grouping filtering on the results.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@9007 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-11 02:00:27 +00:00
Malcolm Tredinnick 61cc4a1eee Fixed #8819 -- Don't include two copies of extra-select columns in the query.
This was triggered by r8794, but was, in fact, fairly fragile before then. The
current fix is the correct way we should be doing this.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@8898 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-03 03:48:25 +00:00
Malcolm Tredinnick 5c32fe7fad Fixed #8790 -- Multi-branch join trees that shared tables of the same name were
sometimes also sharing aliases, instead of creating their own. This was
generating incorrect SQL.

No representative test for this fix yet because I haven't had time to write one
that fits in nicely with the test suite. But it works for the monstrous example
in #8790 and a bunch of other complex examples I've created locally. Will write
a test later.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@8853 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-02 13:52:07 +00:00
Malcolm Tredinnick 12f6259903 Fixed #8439 -- Complex combinations of Q-objects (using both conjunctions and
disjunctions) were producing incorrect SQL when nullable relations were
involved. This fixes that.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@8832 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-02 02:16:41 +00:00
Malcolm Tredinnick 3cdfb47e93 A typo fix that fortunately wasn't creating invalid SQL, just slightly less
efficient than possible SQL in some odd cases (found via code inspection, not
any particular failing example).


git-svn-id: http://code.djangoproject.com/svn/django/trunk@8831 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-02 00:50:19 +00:00
Malcolm Tredinnick 1b615ac46d Get the join promotion code (a bit more) correct when constructing joins for ordering.
Yes, this really is a commit that fixes an oversight in a commit that fixed an
oversight. One day I'll get it right.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@8829 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-02 00:20:10 +00:00
Ian Kelly 6211527fa2 Replaced exact lookups of the empty string with isnull lookups in Oracle. Fixes the testPreventDuplicateComments case.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8827 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-01 23:18:42 +00:00
Russell Keith-Magee f55b834c2f Fixed #7302: Corrected quoting of columns in extra_group_by. Thanks to Ivan Sagalaev for the patch and initial test.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8794 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-01 12:07:26 +00:00
Malcolm Tredinnick 2d7feda05c Fixed #7512 -- Fixed an oversight when I first fixed ordering on nullable
foreign keys (r7761). Thanks to George Vilches for the test case here.


git-svn-id: http://code.djangoproject.com/svn/django/trunk@8783 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2008-09-01 02:43:55 +00:00