Commit Graph

768 Commits

Author SHA1 Message Date
Mariusz Felisiak 93a42d43a6 [4.0.x] Fixed #33159 -- Reverted "Fixed #32970 -- Changed WhereNode.clone() to create a shallow copy of children."
This reverts commit e441847eca.

A shallow copy is not enough because querysets can be reused and
evaluated in nested nodes, which shouldn't mutate JOIN aliases.

Thanks Michal Čihař for the report.
Backport of 903aaa35e5 from main
2021-09-30 11:26:53 +02:00
David Wobrock b2a0978610 [4.0.x] Fixed #33018 -- Fixed annotations with empty queryset.
Thanks Simon Charette for the review and implementation idea.

Backport of dd1fa3a31b from main
2021-09-29 20:53:16 +02:00
David Wobrock aab76433ed [4.0.x] Fixed #33141 -- Renamed Expression.empty_aggregate_value to empty_result_set_value.
Backport of ad36a198a1 from main
2021-09-29 20:52:59 +02:00
Keryn Knight 5353e7c250 Refs #27624 -- Optimized Query.clone() for non-combined queries.
This avoids constructing a generator expression and a new tuple if the
Query has no combined queries.
2021-09-20 12:34:18 +02:00
Keryn Knight e441847eca Fixed #32970 -- Changed WhereNode.clone() to create a shallow copy of children. 2021-09-20 07:03:10 +02:00
Adam Johnson fc91ea1e50 Refs #27624 -- Changed Query.explain_info to namedtuple. 2021-09-17 07:15:12 +02:00
Adam Johnson 2931d847c2 Optimized Query.clone() a bit.
This removes unnecessary "if ... is None" branches, which are already
shallow-copied in the __dict__.copy() call.
2021-09-17 07:12:56 +02:00
David Wobrock 691486a5cf Fixed #33073 -- Fixed queryset crash with aggregation and empty/extra queryset annotation. 2021-09-01 20:59:16 +02:00
Keryn Knight bf5abf1bdc Fixed #33025 -- Avoided accessing the database connections in Query.build_lookup() when not necessary.
Of the built-in backends, only Oracle treats empty strings and nulls as
equal, so avoid testing the default connection backend for
interprets_empty_strings_as_nulls if it can be established from the
lookup that it wouldn't affect the lookup instance returned. This
improves performance a small amount for most lookups being built,
because accessing the connections requires touching the thread critical
`Local` which is an expensive operation.
2021-08-19 13:28:59 +02:00
Mariusz Felisiak b64db05b9c
Removed unnecessary WhereNode.is_summary.
Unnecessary since its introduction in 1df89a60c5.
2021-08-09 19:23:29 +02:00
Keryn Knight 5b8ef8aa5b Refs #32946 -- Changed Query.add_filter() to take two arguments. 2021-07-28 09:38:42 +02:00
Nick Pope c35b81b864 Fixed #32951 -- Removed Query.where_class & co.
Unused since 3caf957ed5.
2021-07-22 08:49:20 +02:00
David Wobrock fd999318ad Fixed #32704 -- Fixed list of deferred fields when chaining QuerySet.defer() after only(). 2021-07-20 12:15:29 +02:00
Keryn Knight 6a970a8b46
Fixed #32944 -- Avoided unnecessary WhereNode.add() calls.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-20 11:59:21 +02:00
Nick Pope 304f6ff46a
Prevented SQLCompiler.execute_sql() from closing cursor twice.
cursor_iter() helper calls cursor.close() in a finally block.
2021-07-16 13:11:46 +02:00
Daniyal f479df7f8d Refs #32508 -- Raised Type/ValueError instead of using "assert" in django.db.models.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-15 11:43:33 +02:00
Ian Foote f42ccdd835 Fixed #27021 -- Allowed lookup expressions in annotations, aggregations, and QuerySet.filter().
Thanks Hannes Ljungberg and Simon Charette for reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-09 11:43:06 +02:00
Wu Haotian aba9c2de66 Fixed #32226 -- Fixed JSON format of QuerySet.explain() on PostgreSQL. 2021-07-05 09:08:39 +02:00
Simon Charette 9f3cce172f Refs #26430 -- Re-introduced empty aggregation optimization.
The introduction of the Expression.empty_aggregate_value interface
allows the compilation stage to enable the EmptyResultSet optimization
if all the aggregates expressions implement it.

This also removes unnecessary RegrCount/Count.convert_value() methods.
Disabling the empty result set aggregation optimization when it wasn't
appropriate prevented None returned for a Count aggregation value.

Thanks Nick Pope for the review.
2021-07-02 07:25:42 +02:00
Simon Charette f3112fde98 Fixed #26430 -- Fixed coalesced aggregation of empty result sets.
Disable the EmptyResultSet optimization when performing aggregation as
it might interfere with coalescence.
2021-07-02 06:40:54 +02:00
Hannes Ljungberg d8c90d4c22 Fixed #32786 -- Moved subquery ordering clearing optimization to the _in lookup.
Co-Authored-By: Simon Charette <charette.s@gmail.com>
2021-06-30 10:08:55 +02:00
Hannes Ljungberg 053141d31f Refs #32786 -- Made Query.clear_ordering() not to cause side effects by default. 2021-06-30 08:52:27 +02:00
Hasan Ramezani 8de4ca74ba Fixed #32693 -- Quoted and lowercased generated column aliases. 2021-05-04 07:36:56 +02:00
Nick Pope 0c71e0f9cf Removed unnecessary reuse_with_filtered_relation argument from Query methods.
In Query.join() the argument reuse_with_filtered_relation was used to
determine whether to use == or .equals(). As this area of code is
related to aliases, we only expect an instance of Join or BaseTable to
be provided - the only two classes that provide .equals().

In both cases, the implementations of __eq__() and equals() are based
on use of the "identity" property. __eq__() performs an isinstance()
check first, returning NotImplemented if required. BaseTable.equals()
then does a straightforward equality check on "identity". Join.equals()
is a little bit different as it skips checking the last element of the
"identity" property: filtered_relation. This was only included
previously when the with_filtered_relation argument was True, impossible
since bbf141bcdc.
2021-04-28 21:19:42 +02:00
Nick Pope a6c79f4987 Removed unused with_filtered_relation argument from .equals()
Unused since bbf141bcdc.
2021-04-28 20:42:20 +02:00
Simon Charette c8b6594305 Fixed #32632, Fixed #32657 -- Removed flawed support for Subquery deconstruction.
Subquery deconstruction support required implementing complex and
expensive equality rules for sql.Query objects for little benefit as
the latter cannot themselves be made deconstructible to their reference
to model classes.

Making Expression @deconstructible and not BaseExpression allows
interested parties to conform to the "expression" API even if they are
not deconstructible as it's only a requirement for expressions allowed
in Model fields and meta options (e.g. constraints, indexes).

Thanks Phillip Cutter for the report.

This also fixes a performance regression in bbf141bcdc.
2021-04-28 12:13:55 +02:00
Mariusz Felisiak 4074f38e1d Refs #32682 -- Fixed QuerySet.delete() crash on querysets with self-referential subqueries on MySQL. 2021-04-27 10:34:47 +02:00
Chris Jerdonek 0461b7a6b6 Fixed #32662 -- Refactored a generator out of SQLCompiler.get_order_by().
This also renames the `asc` variable to `default_order`, markes the
`desc` variable as unused, fixes a typo in SQLCompiler.get_order_by()
docstring, and reorders some blocks in SQLCompiler._order_by_pairs().
2021-04-21 12:35:49 +02:00
Simon Charette 6d0cbe42c3 Fixed #32650 -- Fixed handling subquery aliasing on queryset combination.
This issue started manifesting itself when nesting a combined subquery
relying on exclude() since 8593e162c9 but
sql.Query.combine never properly handled subqueries outer refs in the
first place, see QuerySetBitwiseOperationTests.test_subquery_aliases()
(refs #27149).

Thanks Raffaele Salmaso for the report.
2021-04-21 09:49:15 +02:00
Iuri de Silvio 9760e262f8 Fixed #32627 -- Fixed QuerySet.values()/values_list() crash on combined querysets ordered by unannotated columns. 2021-04-12 21:11:40 +02:00
Adam Johnson d9de74141e Fixed #32442 -- Used converters on returning fields from INSERT statements. 2021-04-07 17:10:00 +02:00
Yuri Konotopov 330bc402a8 Fixed #32546 -- Avoided Meta.ordering columns in GROUP BY clauses.
Follow up to 0ddb4ebf7b.
2021-03-16 07:58:38 +01:00
Mariusz Felisiak ba9a2b7544
Refs #32508 -- Raised TypeError instead of using "assert" on unsupported operations for sliced querysets. 2021-03-10 09:16:28 +01:00
Simon Charette 277eea8fcc Fixed #32478 -- Included nested columns referenced by subqueries in GROUP BY on aggregations.
Regression in fb3f034f1c.

Refs #31094, #31150.

Thanks Igor Pejic for the report.
2021-02-24 09:01:36 +01:00
Mariusz Felisiak 06eec31970 Refs #7098 -- Removed support for passing raw column aliases to order_by().
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Mariusz Felisiak 5e33ec80d1 Refs #30158 -- Made alias argument required in signature of Expression.get_group_by_cols() subclasses.
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Alexander Lyabah 415f50298f Fixed #32231 -- Allowed passing None params to QuerySet.raw(). 2021-01-05 10:37:11 +01:00
Ian Foote 8b040e3cbb Fixed #25534, Fixed #31639 -- Added support for transform references in expressions.
Thanks Mariusz Felisiak and Simon Charette for reviews.
2020-11-27 20:42:04 +01:00
David-Wobrock ba42569d5c Fixed #31507 -- Added QuerySet.exists() optimizations to compound queries. 2020-11-16 20:29:13 +01:00
David-Wobrock 464a4c0c59 Fixed #31496 -- Fixed QuerySet.values()/values_list() crash on combined querysets ordered by annotations. 2020-11-14 10:22:09 +01:00
Simon Charette c2d4926702 Fixed #31910 -- Fixed crash of GIS aggregations over subqueries.
Regression was introduced by fff5186 but was due a long standing issue.

AggregateQuery was abusing Query.subquery: bool by stashing its
compiled inner query's SQL for later use in its compiler which made
select_format checks for Query.subquery wrongly assume the provide
query was a subquery.

This patch prevents that from happening by using a dedicated
inner_query attribute which is compiled at a later time by
SQLAggregateCompiler.

Moving the inner query's compilation to SQLAggregateCompiler.compile
had the side effect of addressing a long standing issue with
aggregation subquery pushdown which prevented converters from being
run. This is now fixed as the aggregation_regress adjustments
demonstrate.

Refs #25367.

Thanks Eran Keydar for the report.
2020-11-04 09:54:58 +01:00
Christian Klus 4ac2d4fa42 Fixed #32152 -- Fixed grouping by subquery aliases.
Regression in 42c08ee465.

Thanks Simon Charette for the review.
2020-10-29 09:56:09 +01:00
Simon Charette 8593e162c9 Fixed #32143 -- Used EXISTS to exclude multi-valued relationships.
As mentioned in the pre-existing split_exclude() docstring EXISTS is
easier to optimize for query planers and circumvents the IN (NULL)
handling issue.
2020-10-28 07:22:00 +01:00
Simon Charette bbf141bcdc Refs #27149 -- Fixed sql.Query identity.
By making Query subclass BaseExpression in
3543129822 the former defined it's
identity based off _construct_args which is not appropriate.
2020-10-28 07:21:53 +01:00
Adam Johnson a56586eafe Fixed #32134 -- Fixed crash of __range lookup with namedtuple.
Regression in 8be79984dc.

Thanks Gordon Wrigley for the report.
2020-10-23 18:01:31 +02:00
Nick Pope 06c5d3fafc Fixed #32060 -- Added Random database function. 2020-10-02 06:58:03 +02:00
Hasan Ramezani a046bcadbe
Fixed #31916 -- Fixed combined queryset crash when combining with ordered combined querysets. 2020-09-17 08:26:06 +02:00
Mariusz Felisiak eaf9764d3b
Fixed #32007 -- Fixed queryset crash with Q() annotation and aggregation.
Thanks Gordon Wrigley for the report.

Regression in 8a6df55f2d.
2020-09-15 11:40:59 +02:00
Simon Charette f6405c0b8e Fixed #31965 -- Adjusted multi-table fast-deletion on MySQL/MariaDB.
The optimization introduced in 7acef095d7 did not properly handle
deletion involving filters against aggregate annotations.

It initially was surfaced by a MariaDB test failure but misattributed
to an undocumented change in behavior that resulted in the systemic
generation of poorly performing database queries in 5b83bae031.

Thanks Anton Plotkin for the report.

Refs #23576.
2020-08-31 08:11:28 +02:00
Simon Charette 51297a9232 Fixed #31792 -- Made Exists() reuse QuerySet.exists() optimizations.
The latter is already optimized to limit the number of results, avoid
selecting unnecessary fields, and drop ordering if possible without
altering the semantic of the query.
2020-08-13 14:10:36 +02:00