Commit Graph

100 Commits

Author SHA1 Message Date
Mariusz Felisiak 93cae5cb2f Fixed CVE-2022-28346 -- Protected QuerySet.annotate(), aggregate(), and extra() against SQL injection in column aliases.
Thanks Splunk team: Preston Elder, Jacob Davis, Jacob Moore,
Matt Hanson, David Briggs, and a security researcher: Danylo Dmytriiev
(DDV_UA) for the report.
2022-04-11 08:59:33 +02:00
Luke Plant 40b8a6174f Fixed #33397 -- Corrected resolving output_field for DateField/DateTimeField/TimeField/DurationFields.
This includes refactoring of CombinedExpression._resolve_output_field()
so it no longer uses the behavior inherited from Expression of guessing
same output type if argument types match, and instead we explicitly
define the output type of all supported operations.

This also makes nonsensical operations involving dates
(e.g. date + date) raise a FieldError, and adds support for
automatically inferring output_field for cases such as:
* date - date
* date + duration
* date - duration
* time + duration
* time - time
2022-03-31 11:05:23 +02:00
Carlton Gibson bb61f0186d Refs #32365 -- Removed internal uses of utils.timezone.utc alias.
Remaining test case ensures that uses of the alias are mapped
canonically by the migration writer.
2022-03-24 06:29:50 +01:00
Nick Pope 847f46e9bf
Removed redundant QuerySet.all() calls in docs and tests.
Most QuerySet methods are mapped onto the Manager and, in general,
it isn't necessary to call .all() on the manager.
2022-02-22 10:29:38 +01:00
Mariusz Felisiak 7119f40c98 Refs #33476 -- Refactored code to strictly match 88 characters line length. 2022-02-07 20:37:05 +01:00
django-bot 9c19aff7c7 Refs #33476 -- Reformatted code with Black. 2022-02-07 20:37:05 +01:00
Mariusz Felisiak 71e7c8e737
Fixed #33468 -- Fixed QuerySet.aggregate() after annotate() crash on aggregates with default.
Thanks Adam Johnson for the report.
2022-01-31 11:33:24 +01:00
Simon Charette e5a92d400a Fixed #33282 -- Fixed a crash when OR'ing subquery and aggregation lookups.
As a QuerySet resolves to Query the outer column references grouping logic
should be defined on the latter and proxied from Subquery for the cases where
get_group_by_cols is called on unresolved expressions.

Thanks Antonio Terceiro for the report and initial patch.
2021-12-02 07:23:33 +01:00
Hannes Ljungberg a934d377af Fixed #33262 -- Fixed crash of conditional aggregation on Exists(). 2021-11-04 06:31:32 +01:00
David Wobrock ad36a198a1 Fixed #33141 -- Renamed Expression.empty_aggregate_value to empty_result_set_value. 2021-09-29 12:58:01 +02:00
David Wobrock 691486a5cf Fixed #33073 -- Fixed queryset crash with aggregation and empty/extra queryset annotation. 2021-09-01 20:59:16 +02:00
Tim Graham 022d29c934
Refs #10929 -- Allowed NowUTC SQL customization for third-party backends. 2021-08-24 08:28:03 +02:00
Nick Pope 501a8db465 Fixed #10929 -- Added default argument to aggregates.
Thanks to Simon Charette and Adam Johnson for the reviews.
2021-07-19 13:04:27 +02:00
Simon Charette 9f3cce172f Refs #26430 -- Re-introduced empty aggregation optimization.
The introduction of the Expression.empty_aggregate_value interface
allows the compilation stage to enable the EmptyResultSet optimization
if all the aggregates expressions implement it.

This also removes unnecessary RegrCount/Count.convert_value() methods.
Disabling the empty result set aggregation optimization when it wasn't
appropriate prevented None returned for a Count aggregation value.

Thanks Nick Pope for the review.
2021-07-02 07:25:42 +02:00
Simon Charette f3112fde98 Fixed #26430 -- Fixed coalesced aggregation of empty result sets.
Disable the EmptyResultSet optimization when performing aggregation as
it might interfere with coalescence.
2021-07-02 06:40:54 +02:00
Simon Charette 277eea8fcc Fixed #32478 -- Included nested columns referenced by subqueries in GROUP BY on aggregations.
Regression in fb3f034f1c.

Refs #31094, #31150.

Thanks Igor Pejic for the report.
2021-02-24 09:01:36 +01:00
Hasan Ramezani 275dd4ebba
Fixed #32178 -- Allowed database backends to skip tests and mark expected failures.
Co-authored-by: Tim Graham <timograham@gmail.com>
2020-12-10 18:00:57 +01:00
Ian Foote 8b040e3cbb Fixed #25534, Fixed #31639 -- Added support for transform references in expressions.
Thanks Mariusz Felisiak and Simon Charette for reviews.
2020-11-27 20:42:04 +01:00
Hasan Ramezani 3f7b327562 Fixed #31235 -- Made assertQuerysetEqual() compare querysets directly.
This also replaces assertQuerysetEqual() to
assertSequenceEqual()/assertCountEqual() where appropriate.

Co-authored-by: Peter Inglesby <peter.inglesby@gmail.com>
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2020-11-06 09:24:50 +01:00
Sicong 09e1ec71df
Fixed #32166 -- Removed redundant definition of Greatest in test_expression_on_aggregation. 2020-11-03 07:08:42 +01:00
Étienne Beaulé 509d9da26f Fixed #26390 -- Disabled grouping by Random().
Thanks to Tzu-ping Chung for the tests.
2020-10-21 20:54:53 +02:00
David Wobrock 058747b77a Fixed #31880 -- Made QuerySet.aggregate() raise FieldError when aggregating over aggregation aliases. 2020-09-29 09:04:25 +02:00
Ahmad A. Hussein 493b26bbfc Fixed #31888 -- Avoided module-level MySQL queries in tests. 2020-08-17 09:31:16 +02:00
Simon Charette 156a2138db Refs #30446 -- Removed unnecessary Value(..., output_field) in docs and tests. 2020-07-15 10:58:38 +02:00
Simon Charette 1e38f1191d Fixed #30446 -- Resolved Value.output_field for stdlib types.
This required implementing a limited form of dynamic dispatch to combine
expressions with numerical output. Refs #26355 should eventually provide
a better interface for that.
2020-07-15 10:58:29 +02:00
Simon Charette adfbf653dc Fixed #31568 -- Fixed alias reference when aggregating over multiple subqueries.
691def10a0 made all Subquery() instances
equal to each other which broke aggregation subquery pushdown which
relied on object equality to determine which alias it should select.

Subquery.__eq__() will be fixed in an another commit but
Query.rewrite_cols() should haved used object identity from the start.

Refs #30727, #30188.

Thanks Makina Corpus for the report.
2020-05-14 09:31:15 +02:00
Hasan Ramezani 10866a10fe Fixed #31377 -- Disabled grouping by aliases on QuerySet.values()/values_list() when they collide with field names.
Regression in fb3f034f1c.

Thanks Holovashchenko Vadym for the report.
2020-03-25 09:13:46 +01:00
Mariusz Felisiak aee0bebc2f Refs #31331 -- Checked ONLY_FULL_GROUP_BY mode in AggregateTestCase.test_aggregation_subquery_annotation_multivalued(). 2020-03-03 11:25:37 +01:00
Mariusz Felisiak 7b8fa1653f Fixed #31150 -- Included subqueries that reference related fields in GROUP BY clauses.
Thanks Johannes Hoppe for the report.

Regression in fb3f034f1c.

Co-authored-by: Simon Charette <charette.s@gmail.com>
2020-03-03 11:25:37 +01:00
Nick Pope 335c9c94ac Simplified imports from django.db and django.contrib.gis.db. 2020-02-04 13:20:06 +01:00
Mariusz Felisiak 6b178a3e93
Fixed #31217 -- Made QuerySet.values()/values_list() group by not selected annotations with aggregations used in order_by().
Regression in 59b4e99dd0.

Thanks Jon Dufresne for the report and Simon Charette for the review.
2020-02-03 07:48:11 +01:00
Mariusz Felisiak 59b4e99dd0
Refs #31136 -- Made QuerySet.values()/values_list() group only by selected annotation.
Regression in 0f843fdd5b.
2020-01-15 09:32:42 +01:00
Mariusz Felisiak 0f843fdd5b
Fixed #31136 -- Disabled grouping by aliases on QuerySet.values()/values_list().
Regression in fb3f034f1c.

Thanks Sigurd Ljødal for the report.
2020-01-04 20:49:11 +01:00
Simon Charette 720de4d044 Fixed #31109 -- Disabled grouping by aliases on QuerySet.exists().
Clearing the SELECT clause in Query.has_results was orphaning GROUP BY
references to it.

Thanks Thierry Bastian for the report and Baptiste Mispelon for the
bisect.

Regression in fb3f034f1c.
2019-12-23 09:24:03 +01:00
Simon Charette 5a4d7285bd Fixed #31094 -- Included columns referenced by subqueries in GROUP BY on aggregations.
Thanks Johannes Hoppe for the report.

Regression in fb3f034f1c.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2019-12-19 10:04:56 +01:00
Simon Charette a0f34d8fef Refs #31094 -- Added test for not using aliases in GROUP BY clauses. 2019-12-19 09:12:29 +01:00
Tim Graham 9100c664db Relaxed some query ordering assertions in tests.
It accounts for differences seen on cockroachdb.
2019-11-18 12:32:37 +01:00
Étienne Beaulé 5f24e7158e Fixed #30665 -- Added support for distinct argument to Avg() and Sum(). 2019-07-31 11:22:50 +02:00
Étienne Beaulé cb3c2da128 Moved test for distinct Count() to a separate test case. 2019-07-31 10:41:17 +02:00
CruxBox 34f8eeea4a Fixed #30548 -- Improved exception when expression contains mixed types. 2019-06-10 19:48:53 +02:00
Étienne Beaulé 4b6dfe1622 Fixed #30542 -- Fixed crash of numerical aggregations with filter.
Filters in annotations crashed when used with numerical-type
aggregations (i.e. Avg, StdDev, and Variance). This was caused as the
source expressions no not necessarily have an output_field (such as the
filter field), which lead to an AttributeError: 'WhereNode' object has
no attribute output_field.

Thanks to Chuan-Zheng Lee for the report.

Regression in c690afb873 and two following
commits.
2019-06-05 08:06:26 +02:00
Mariusz Felisiak ef9f2eb69c
Refs #30462 -- Added test for conditional aggregation on annotated subquery. 2019-05-08 11:29:17 +02:00
Simon Charette 1ca825e4dc Fixed #30246 -- Reused annotation aliases references in aggregation filters.
Thanks Jan Baryła for the detailed report and the reduced test case.
2019-03-23 10:11:45 -04:00
Simon Charette fb3f034f1c Fixed #30158 -- Avoided unnecessary subquery group by on aggregation.
Subquery annotations can be omitted from the GROUP BY clause on aggregation
as long as they are not explicitly grouped against.

Thanks Jonny Fuller for the report.
2019-03-21 18:48:41 -04:00
Simon Charette 65858119d2 Fixed #30120 -- Fixed invalid SQL in distinct aggregate.
Regression in bc05547cd8 (refs #28658).
2019-01-21 09:24:32 -05:00
Nick Pope c690afb873 Refs #28643 -- Changed Avg() to use NumericOutputFieldMixin.
Keeps precision instead of forcing DecimalField to FloatField.
2019-01-14 14:35:41 -05:00
Simon Charette bc05547cd8 Fixed #28658 -- Added DISTINCT handling to the Aggregate class. 2019-01-09 17:52:36 -05:00
Simon Charette 53269bcaaf Fixed #30011 -- Fixed queries that reuse filtered aggregates.
Thanks Taqi Abbas and Raphael Kimmig for the report.
2018-12-06 14:55:35 -05:00
priyanshsaxena 83b04d4f88 Fixed #29048 -- Added **extra_context to database function as_vendor() methods. 2018-08-23 11:52:09 -04:00
Mariusz Felisiak 75e45a45e7 Simplified aggregation.tests. 2018-02-02 08:47:32 +01:00