Thanks Josh Smeaton, Mariusz Felisiak, Sergey Fedoseev, Simon Charettes,
Adam Chainz/Johnson and Tim Graham for comments and reviews and Jamie
Cockburn for initial patch.
Instead of splitting filter clauses to where and having parts before
adding them to query.where or query.having, add all filter clauses to
query.where, and when compiling the query split the where to having and
where parts.
Allowed users to specify which lookups or transforms ("nested lookus")
are available for fields. The implementation is now class based.
Squashed commit of the following:
commit fa7a7195f1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 18 10:53:24 2014 +0200
Added lookup registration API docs
commit eb1c8ce164
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Tue Jan 14 18:59:36 2014 +0200
Release notes and other minor docs changes
commit 11501c29c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 20:53:03 2014 +0200
Forgot to add custom_lookups tests in prev commit
commit 83173b960e
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:59:12 2014 +0200
Renamed Extract -> Transform
commit 3b18d9f3a1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:51:53 2014 +0200
Removed suggestion of temporary lookup registration from docs
commit 21d0c7631c
Merge: 2509006f2dc442
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 09:38:23 2014 -0800
Merge pull request #2 from mjtamlyn/lookups_3
Reworked custom lookups docs.
commit f2dc4429a1
Author: Marc Tamlyn <marc.tamlyn@gmail.com>
Date: Sun Jan 12 13:15:05 2014 +0000
Reworked custom lookups docs.
Mostly just formatting and rewording, but also replaced the example
using ``YearExtract`` to use an example which is unlikely to ever be
possible directly in the ORM.
commit 2509006506
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 13:19:13 2014 +0200
Removed unused import
commit 4fba5dfaa0
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:34:41 2014 +0200
Added docs to index
commit 6d53963f37
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:10:24 2014 +0200
Dead code removal
commit f9cc039007
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 19:00:43 2014 +0200
A new try for docs
commit 33aa18a6e3
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:57:12 2014 +0200
Renamed get_cols to get_group_by_cols
commit c7d5f8661b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:45:53 2014 +0200
Altered query string customization for backends vendors
The new way is trying to call first method 'as_' + connection.vendor.
If that doesn't exist, then call as_sql().
Also altered how lookup registration is done. There is now
RegisterLookupMixin class that is used by Field, Extract and
sql.Aggregate. This allows one to register lookups for extracts and
aggregates in the same way lookup registration is done for fields.
commit 90e7004ec1
Merge: 66649fff7c2c0a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:21:01 2014 +0200
Merge branch 'master' into lookups_3
commit 66649ff891
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:16:01 2014 +0200
Some rewording in docs
commit 31b8faa627
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 29 15:52:29 2013 +0200
Cleanup based on review comments
commit 1016159f34
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 18:37:04 2013 +0200
Proof-of-concept fix for #16731
Implemented only for SQLite and PostgreSQL, and only for startswith
and istartswith lookups.
commit 193cd097ca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 17:57:58 2013 +0200
Fixed#11722 -- iexact=F() produced invalid SQL
commit 08ed3c3b49
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:59:52 2013 +0200
Made Lookup and Extract available from django.db.models
commit b99c8d83c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:06:29 2013 +0200
Fixed review notes by Loic
commit 049eebc070
Merge: ed8fab7b80a835
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:53:10 2013 +0200
Merge branch 'master' into lookups_3
Conflicts:
django/db/models/fields/__init__.py
django/db/models/sql/compiler.py
django/db/models/sql/query.py
tests/null_queries/tests.py
commit ed8fab7fe8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:47:23 2013 +0200
Made Extracts aware of full lookup path
commit 27a57b7aed
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:10:11 2013 +0200
Removed debugger import
commit 074e0f5aca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:02:16 2013 +0200
GIS lookup support added
commit 760e28e72b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 20:04:31 2013 +0200
Removed usage of Constraint, used Lookup instead
commit eac4776684
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:22:30 2013 +0200
Minor cleanup of Lookup API
commit 2adf50428d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:14:19 2013 +0200
Added documentation, polished implementation
commit 32c04357a8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:10:15 2013 +0200
Avoid OrderedDict creation on lookup aggregate check
commit 7c8b3a32cc
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:04:34 2013 +0200
Implemented nested lookups
But there is no support of using lookups outside filtering yet.
commit 4d219d4cde
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Wed Nov 27 22:07:30 2013 +0200
Initial implementation of custom lookups
The SubqueryConstraint defined relabeled_clone(), but that was never
called. Instead there is now clone() and relabel_aliases() methods for
SubqueryConstraint.
A related problem was that SubqueryConstraint didn't correctly use
quote_name_unless_alias() of the outer query. This resulted in failures
when running under PostgreSQL.
The sql/query.py add_q method did a lot of where/having tree hacking to
get complex queries to work correctly. The logic was refactored so that
it should be simpler to understand. The new logic should also produce
leaner WHERE conditions.
The changes cascade somewhat, as some other parts of Django (like
add_filter() and WhereNode) expect boolean trees in certain format or
they fail to work. So to fix the add_q() one must fix utils/tree.py,
some things in add_filter(), WhereNode and so on.
This commit also fixed add_filter to see negate clauses up the path.
A query like .exclude(Q(reversefk__in=a_list)) didn't work similarly to
.filter(~Q(reversefk__in=a_list)). The reason for this is that only
the immediate parent negate clauses were seen by add_filter, and thus a
tree like AND: (NOT AND: (AND: condition)) will not be handled
correctly, as there is one intermediary AND node in the tree. The
example tree is generated by .exclude(~Q(reversefk__in=a_list)).
Still, aggregation lost connectors in OR cases, and F() objects and
aggregates in same filter clause caused GROUP BY problems on some
databases.
Fixed#17600, fixed#13198, fixed#17025, fixed#17000, fixed#11293.
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
Thanks Carl Meyer for the review.
Squashed commit of the following:
commit 4f290bdb60
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 21:21:30 2013 +0100
Used '0:00' instead of 'UTC' which doesn't always exist in Oracle.
Thanks Ian Kelly for the suggestion.
commit 01b6366f3c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 13:38:43 2013 +0100
Made tzname a parameter of datetime_extract/trunc_sql.
This is required to work around a bug in Oracle.
commit 924a144ef8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 14:47:44 2013 +0100
Added support for parameters in SELECT clauses.
commit b4351d2890
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 22:30:22 2013 +0100
Documented backwards incompatibilities in the two previous commits.
commit 91ef84713c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:42:31 2013 +0100
Used QuerySet.datetimes for the admin's date_hierarchy.
commit 0d0de288a5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:29:38 2013 +0100
Used QuerySet.datetimes in date-based generic views.
commit 9c0859ff7c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:25 2013 +0100
Implemented QuerySet.datetimes on Oracle.
commit 68ab511a4f
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:14 2013 +0100
Implemented QuerySet.datetimes on MySQL.
commit 22d52681d3
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:42:29 2013 +0100
Implemented QuerySet.datetimes on SQLite.
commit f6800fd04c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:03 2013 +0100
Implemented QuerySet.datetimes on PostgreSQL.
commit 0c829c23f4
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:41:08 2013 +0100
Added datetime-handling infrastructure in the ORM layers.
commit 104d82a777
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 10:05:55 2013 +0100
Updated null_queries tests to avoid clashing with the __second lookup.
commit c01bbb3235
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 23:07:41 2013 +0100
Updated tests of .dates().
Replaced .dates() by .datetimes() for DateTimeFields.
Replaced dates with datetimes in the expected output for DateFields.
commit 50fb7a5246
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:40:09 2013 +0100
Updated and added tests for QuerySet.datetimes.
commit a8451a5004
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 22:34:46 2013 +0100
Documented the new time lookups and updated the date lookups.
commit 29413eab2b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 16:15:49 2013 +0100
Documented QuerySet.datetimes and updated QuerySet.dates.
value_annotation isn't very well defined. Before this change, setting it
to datetime.datetime could silently reverse the behavior of isnull
lookups. This commit doesn't have any consequences on the current code.
It's just a safeguard for future ORM hackers.
The original problem was that queryset cloning was really expensive
when filtering with F() clauses. The __deepcopy__ went too deep copying
_meta attributes of the models used. To fix this the use of
__deepcopy__ in qs cloning was removed.
This commit results in some speed improvements across the djangobench
benchmark suite. Most query_* tests are 20-30% faster, save() is 50%
faster and finally complex filtering situations can see 2x to order
of magnitude improvments.
Thanks to Suor, Alex and lrekucki for valuable feedback.
The guarantee that no queries will be made when accessing results is
done by new EmptyWhere class which is used for query.where and having.
Thanks to Simon Charette for reviewing and valuable suggestions.
At least Oracle needs parentheses in negated where conditions, even if
there is only single condition negated. Fixed this by reverting to old
logic in that part of as_sql() and adding a comment about this.
I did not investigate why the parentheses are needed. The original
offending commit was bd283aa844.
Made sure the WhereNode.as_sql() handles various EmptyResultSet and
FullResultSet conditions correctly. Also, got rid of the FullResultSet
exception class. It is now represented by '', [] return value in the
as_sql() methods.
Simon Charette
Sergey Fedoseev
Mads Jensen
Anton Samarchyan
Josh Smeaton
Tim Graham
Johannes Dollinger
Anssi Kääriäinen
Anssi Kääriäinen
Loic Bistuer
Simon Charette
Michał Modzelewski
Michael Hall
Alex Gaynor
Justin Hamade
Claude Paroz
Christopher Medrela
Erik Romijn
Marc Tamlyn
Honza Kral
Marcin Biernat
Aymeric Augustin
Julien Phalip