Commit Graph

480 Commits

Author SHA1 Message Date
David Wobrock b2a0978610 [4.0.x] Fixed #33018 -- Fixed annotations with empty queryset.
Thanks Simon Charette for the review and implementation idea.

Backport of dd1fa3a31b from main
2021-09-29 20:53:16 +02:00
David Wobrock aab76433ed [4.0.x] Fixed #33141 -- Renamed Expression.empty_aggregate_value to empty_result_set_value.
Backport of ad36a198a1 from main
2021-09-29 20:52:59 +02:00
Keryn Knight 5353e7c250 Refs #27624 -- Optimized Query.clone() for non-combined queries.
This avoids constructing a generator expression and a new tuple if the
Query has no combined queries.
2021-09-20 12:34:18 +02:00
Adam Johnson fc91ea1e50 Refs #27624 -- Changed Query.explain_info to namedtuple. 2021-09-17 07:15:12 +02:00
Adam Johnson 2931d847c2 Optimized Query.clone() a bit.
This removes unnecessary "if ... is None" branches, which are already
shallow-copied in the __dict__.copy() call.
2021-09-17 07:12:56 +02:00
David Wobrock 691486a5cf Fixed #33073 -- Fixed queryset crash with aggregation and empty/extra queryset annotation. 2021-09-01 20:59:16 +02:00
Keryn Knight bf5abf1bdc Fixed #33025 -- Avoided accessing the database connections in Query.build_lookup() when not necessary.
Of the built-in backends, only Oracle treats empty strings and nulls as
equal, so avoid testing the default connection backend for
interprets_empty_strings_as_nulls if it can be established from the
lookup that it wouldn't affect the lookup instance returned. This
improves performance a small amount for most lookups being built,
because accessing the connections requires touching the thread critical
`Local` which is an expensive operation.
2021-08-19 13:28:59 +02:00
Keryn Knight 5b8ef8aa5b Refs #32946 -- Changed Query.add_filter() to take two arguments. 2021-07-28 09:38:42 +02:00
Nick Pope c35b81b864 Fixed #32951 -- Removed Query.where_class & co.
Unused since 3caf957ed5.
2021-07-22 08:49:20 +02:00
David Wobrock fd999318ad Fixed #32704 -- Fixed list of deferred fields when chaining QuerySet.defer() after only(). 2021-07-20 12:15:29 +02:00
Keryn Knight 6a970a8b46
Fixed #32944 -- Avoided unnecessary WhereNode.add() calls.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-20 11:59:21 +02:00
Daniyal f479df7f8d Refs #32508 -- Raised Type/ValueError instead of using "assert" in django.db.models.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-15 11:43:33 +02:00
Ian Foote f42ccdd835 Fixed #27021 -- Allowed lookup expressions in annotations, aggregations, and QuerySet.filter().
Thanks Hannes Ljungberg and Simon Charette for reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-09 11:43:06 +02:00
Simon Charette 9f3cce172f Refs #26430 -- Re-introduced empty aggregation optimization.
The introduction of the Expression.empty_aggregate_value interface
allows the compilation stage to enable the EmptyResultSet optimization
if all the aggregates expressions implement it.

This also removes unnecessary RegrCount/Count.convert_value() methods.
Disabling the empty result set aggregation optimization when it wasn't
appropriate prevented None returned for a Count aggregation value.

Thanks Nick Pope for the review.
2021-07-02 07:25:42 +02:00
Simon Charette f3112fde98 Fixed #26430 -- Fixed coalesced aggregation of empty result sets.
Disable the EmptyResultSet optimization when performing aggregation as
it might interfere with coalescence.
2021-07-02 06:40:54 +02:00
Hannes Ljungberg d8c90d4c22 Fixed #32786 -- Moved subquery ordering clearing optimization to the _in lookup.
Co-Authored-By: Simon Charette <charette.s@gmail.com>
2021-06-30 10:08:55 +02:00
Hannes Ljungberg 053141d31f Refs #32786 -- Made Query.clear_ordering() not to cause side effects by default. 2021-06-30 08:52:27 +02:00
Nick Pope 0c71e0f9cf Removed unnecessary reuse_with_filtered_relation argument from Query methods.
In Query.join() the argument reuse_with_filtered_relation was used to
determine whether to use == or .equals(). As this area of code is
related to aliases, we only expect an instance of Join or BaseTable to
be provided - the only two classes that provide .equals().

In both cases, the implementations of __eq__() and equals() are based
on use of the "identity" property. __eq__() performs an isinstance()
check first, returning NotImplemented if required. BaseTable.equals()
then does a straightforward equality check on "identity". Join.equals()
is a little bit different as it skips checking the last element of the
"identity" property: filtered_relation. This was only included
previously when the with_filtered_relation argument was True, impossible
since bbf141bcdc.
2021-04-28 21:19:42 +02:00
Nick Pope a6c79f4987 Removed unused with_filtered_relation argument from .equals()
Unused since bbf141bcdc.
2021-04-28 20:42:20 +02:00
Simon Charette c8b6594305 Fixed #32632, Fixed #32657 -- Removed flawed support for Subquery deconstruction.
Subquery deconstruction support required implementing complex and
expensive equality rules for sql.Query objects for little benefit as
the latter cannot themselves be made deconstructible to their reference
to model classes.

Making Expression @deconstructible and not BaseExpression allows
interested parties to conform to the "expression" API even if they are
not deconstructible as it's only a requirement for expressions allowed
in Model fields and meta options (e.g. constraints, indexes).

Thanks Phillip Cutter for the report.

This also fixes a performance regression in bbf141bcdc.
2021-04-28 12:13:55 +02:00
Simon Charette 6d0cbe42c3 Fixed #32650 -- Fixed handling subquery aliasing on queryset combination.
This issue started manifesting itself when nesting a combined subquery
relying on exclude() since 8593e162c9 but
sql.Query.combine never properly handled subqueries outer refs in the
first place, see QuerySetBitwiseOperationTests.test_subquery_aliases()
(refs #27149).

Thanks Raffaele Salmaso for the report.
2021-04-21 09:49:15 +02:00
Mariusz Felisiak ba9a2b7544
Refs #32508 -- Raised TypeError instead of using "assert" on unsupported operations for sliced querysets. 2021-03-10 09:16:28 +01:00
Simon Charette 277eea8fcc Fixed #32478 -- Included nested columns referenced by subqueries in GROUP BY on aggregations.
Regression in fb3f034f1c.

Refs #31094, #31150.

Thanks Igor Pejic for the report.
2021-02-24 09:01:36 +01:00
Mariusz Felisiak 06eec31970 Refs #7098 -- Removed support for passing raw column aliases to order_by().
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Mariusz Felisiak 5e33ec80d1 Refs #30158 -- Made alias argument required in signature of Expression.get_group_by_cols() subclasses.
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Alexander Lyabah 415f50298f Fixed #32231 -- Allowed passing None params to QuerySet.raw(). 2021-01-05 10:37:11 +01:00
Ian Foote 8b040e3cbb Fixed #25534, Fixed #31639 -- Added support for transform references in expressions.
Thanks Mariusz Felisiak and Simon Charette for reviews.
2020-11-27 20:42:04 +01:00
David-Wobrock ba42569d5c Fixed #31507 -- Added QuerySet.exists() optimizations to compound queries. 2020-11-16 20:29:13 +01:00
David-Wobrock 464a4c0c59 Fixed #31496 -- Fixed QuerySet.values()/values_list() crash on combined querysets ordered by annotations. 2020-11-14 10:22:09 +01:00
Simon Charette c2d4926702 Fixed #31910 -- Fixed crash of GIS aggregations over subqueries.
Regression was introduced by fff5186 but was due a long standing issue.

AggregateQuery was abusing Query.subquery: bool by stashing its
compiled inner query's SQL for later use in its compiler which made
select_format checks for Query.subquery wrongly assume the provide
query was a subquery.

This patch prevents that from happening by using a dedicated
inner_query attribute which is compiled at a later time by
SQLAggregateCompiler.

Moving the inner query's compilation to SQLAggregateCompiler.compile
had the side effect of addressing a long standing issue with
aggregation subquery pushdown which prevented converters from being
run. This is now fixed as the aggregation_regress adjustments
demonstrate.

Refs #25367.

Thanks Eran Keydar for the report.
2020-11-04 09:54:58 +01:00
Christian Klus 4ac2d4fa42 Fixed #32152 -- Fixed grouping by subquery aliases.
Regression in 42c08ee465.

Thanks Simon Charette for the review.
2020-10-29 09:56:09 +01:00
Simon Charette 8593e162c9 Fixed #32143 -- Used EXISTS to exclude multi-valued relationships.
As mentioned in the pre-existing split_exclude() docstring EXISTS is
easier to optimize for query planers and circumvents the IN (NULL)
handling issue.
2020-10-28 07:22:00 +01:00
Simon Charette bbf141bcdc Refs #27149 -- Fixed sql.Query identity.
By making Query subclass BaseExpression in
3543129822 the former defined it's
identity based off _construct_args which is not appropriate.
2020-10-28 07:21:53 +01:00
Adam Johnson a56586eafe Fixed #32134 -- Fixed crash of __range lookup with namedtuple.
Regression in 8be79984dc.

Thanks Gordon Wrigley for the report.
2020-10-23 18:01:31 +02:00
Simon Charette 51297a9232 Fixed #31792 -- Made Exists() reuse QuerySet.exists() optimizations.
The latter is already optimized to limit the number of results, avoid
selecting unnecessary fields, and drop ordering if possible without
altering the semantic of the query.
2020-08-13 14:10:36 +02:00
Alexandr Tatarinov f4ac167119 Fixed #27719 -- Added QuerySet.alias() to allow creating reusable aliases.
QuerySet.alias() allows creating reusable aliases for expressions that
don't need to be selected but are used for filtering, ordering, or as
a part of complex expressions.

Thanks Simon Charette for reviews.
2020-07-31 13:19:33 +02:00
matt ferrante 7d6916e827 Fixed #29789 -- Added support for nested relations to FilteredRelation. 2020-07-09 20:24:00 +02:00
Mariusz Felisiak ae8338daf3
Fixed #31767 -- Fixed QuerySet.none() on combined queryset. 2020-07-07 12:15:05 +02:00
Jacob Walls 512da9d585 Fixed #23797 -- Fixed QuerySet.exclude() when rhs is a nullable column. 2020-07-06 10:15:37 +02:00
Nicolas Baccelli b38d44229f Fixed #31664 -- Reallowed using non-expressions having filterable attribute as rhs in queryset filters.
Regression in 4edad1ddf6.
2020-06-08 08:17:23 +02:00
Manuel Weitzman a4e6030904 Fixed #30375 -- Added FOR NO KEY UPDATE support to QuerySet.select_for_update() on PostgreSQL. 2020-05-21 10:51:10 +02:00
Simon Charette adfbf653dc Fixed #31568 -- Fixed alias reference when aggregating over multiple subqueries.
691def10a0 made all Subquery() instances
equal to each other which broke aggregation subquery pushdown which
relied on object equality to determine which alias it should select.

Subquery.__eq__() will be fixed in an another commit but
Query.rewrite_cols() should haved used object identity from the start.

Refs #30727, #30188.

Thanks Makina Corpus for the report.
2020-05-14 09:31:15 +02:00
Simon Charette 42c08ee465 Fixed #31566 -- Fixed aliases crash when chaining values()/values_list() after annotate() with aggregations and subqueries.
Subquery annotation references must be resolved if they are excluded
from the GROUP BY clause by a following .values() call.

Regression in fb3f034f1c.

Thanks Makina Corpus for the report.
2020-05-14 08:16:16 +02:00
Simon Charette 513948735b Fixed #31426 -- Added proper field validation to QuerySet.order_by().
Resolve the field reference instead of using fragile regex based string
reference validation.
2020-04-06 10:19:49 +02:00
Simon Charette 98ea4f0f46 Refs #7098 -- Deprecated passing raw column aliases to order_by().
Now that order_by() has expression support passing RawSQL() can achieve
the same result.

This was also already supported through QuerySet.extra(order_by) for
years but this API is more or less deprecated at this point.
2020-04-06 10:01:02 +02:00
Hasan Ramezani 4237050684 Fixed #31420 -- Fixed crash when filtering subquery annotation against a SimpleLazyObject.
Thanks Simon Charette for the solution and analysis.
2020-04-06 07:11:29 +02:00
Hasan Ramezani 10866a10fe Fixed #31377 -- Disabled grouping by aliases on QuerySet.values()/values_list() when they collide with field names.
Regression in fb3f034f1c.

Thanks Holovashchenko Vadym for the report.
2020-03-25 09:13:46 +01:00
Mariusz Felisiak 59b4e99dd0
Refs #31136 -- Made QuerySet.values()/values_list() group only by selected annotation.
Regression in 0f843fdd5b.
2020-01-15 09:32:42 +01:00
Mariusz Felisiak 0f843fdd5b
Fixed #31136 -- Disabled grouping by aliases on QuerySet.values()/values_list().
Regression in fb3f034f1c.

Thanks Sigurd Ljødal for the report.
2020-01-04 20:49:11 +01:00
Simon Charette 720de4d044 Fixed #31109 -- Disabled grouping by aliases on QuerySet.exists().
Clearing the SELECT clause in Query.has_results was orphaning GROUP BY
references to it.

Thanks Thierry Bastian for the report and Baptiste Mispelon for the
bisect.

Regression in fb3f034f1c.
2019-12-23 09:24:03 +01:00