Commit Graph

5251 Commits

Author SHA1 Message Date
Simon Charette c6350d594c Refs #30158 -- Removed alias argument for Expression.get_group_by_cols().
Recent refactors allowed GROUP BY aliasing allowed for aliasing to be
entirely handled by the sql.Query.set_group_by and compiler layers.
2022-10-06 12:04:00 +02:00
Simon Charette b7b28c7c18 Refs #31150 -- Enabled implicit GROUP BY aliases.
This ensures implicit grouping from aggregate function annotations
groups by uncollapsed selected aliases if supported.

The feature is disabled on Oracle because it doesn't support it.
2022-10-06 11:51:32 +02:00
Simon Charette 3d734c09ff Refs #33992 -- Refactored subquery grouping logic.
This required moving the combined queries slicing logic to the compiler
in order to allow Query.exists() to be called at expression resolving
time.

It allowed for Query.exists() to be called at Exists() initialization
time and thus ensured that get_group_by_cols() was operating on the
terminal representation of the query that only has a single column
selected.
2022-10-06 11:38:03 +02:00
Simon Charette 04518e310d Refs #33308 -- Enabled explicit GROUP BY and ORDER BY aliases.
This ensures explicit grouping from using values() before annotating an
aggregate function groups by selected aliases if supported.

The GROUP BY feature is disabled on Oracle because it doesn't support it.
2022-10-06 09:34:31 +02:00
Simon Charette c58a8acd41 Fixed #33768 -- Fixed ordering compound queries by nulls_first/nulls_last on MySQL.
Columns of the left outer most select statement in a combined query
can be referenced by alias just like by index.

This removes combined query ordering by column index and avoids an
unnecessary usage of RawSQL which causes issues for backends that
specialize the treatment of null ordering.
2022-10-05 14:04:49 +02:00
Simon Charette a142edcc49 Avoided unnecessary call to .get_source_expressions().
The SQLCompiler._order_by_pairs() generator method yields instances of
OrderBy and not Expression.
2022-10-04 12:29:37 +02:00
David Sanders 4771a1694b Fixed #34012 -- Made QuerySet.order_by() apply transforms on related fields for models with Meta.ordering.
This makes QuerySet.order_by() no longer ignore trailing transforms for
models with Meta.ordering. As a consequence, FieldError is raised in
such cases for non-existent fields.

Thanks to Klaas van Schelven for the report and Mariusz Felisiak for the
review and advice.
2022-10-04 08:19:34 +02:00
Lily Foote 649b28eab6 Fixed #34070 -- Added subsecond support to Now() on SQLite and MySQL. 2022-10-03 12:13:27 +02:00
David Wobrock 3b4a5b9f97 Fixed #23353 -- Used "raise from" when raising TransactionManagementError.
This change sets the __cause__ attribute to raised exceptions.
2022-10-03 06:08:54 +02:00
Mariusz Felisiak bc3b8f1524
Refs #34058 -- Fixed changing/deleting sequences when altering pre-Django 4.1 auto fields on PostgreSQL.
Thanks Anders Kaseorg for the report.

Follow up to 19e6efa50b.
Regression in 2eea361eff.
2022-10-01 07:53:32 +02:00
Mariusz Felisiak 5e0aa362d9
Fixed #33984 -- Reverted "Fixed #32980 -- Made models cache related managers."
This reverts 4f8c7fd9d9 and adds
two regression tests:
- test_related_manager_refresh(), and
- test_create_copy_with_m2m().

Thanks joeli for the report.
2022-09-30 18:18:33 +02:00
sarahboyce 6cc0f22a73 Fixed #32095 -- Made QuerySet.update_or_create() save only fields passed in defaults or with custom pre_save().
Thanks Florian Apolloner for the initial patch.
2022-09-30 06:26:19 +02:00
sarahboyce 1d77b931f7 Refs #32095 -- Added model's Options._non_pk_concrete_field_names property. 2022-09-29 20:32:08 +02:00
Mariusz Felisiak 19e6efa50b
Fixed #34058 -- Changed sequence types when altering pre-Django 4.1 auto fields on PostgreSQL.
Thanks Anders Kaseorg for the report.

Thanks Florian Apolloner for pair programming.

Regression in 2eea361eff.
2022-09-29 13:20:14 +02:00
David Wobrock cff1f888e9
Fixed #33464 -- Resolved output_field for combined numeric expressions with MOD operator. 2022-09-27 20:41:10 +02:00
Adam Johnson cd03e8e2d6 Fixed #34050 -- Replaced invalid chars in migration names with '_'.
Thanks to Bishal Gautam for the report and initial implementation.

Regression in fa58450a9a.

Co-Authored-By: Bishal Gautam <bisalgt@gmail.com>
2022-09-27 08:02:51 +02:00
David Sanders 9f8c994851 Fixed #34027 -- Fixed migrations crash when altering type of char/text fields referenced by foreign key on PostgreSQL. 2022-09-25 20:23:01 +02:00
David Sanders 1674c70525
Fixed #34024 -- Fixed crash when aggregating querysets with Q objects annotations.
This reverts b64db05b9c.

It was reasonable to assume it was unnecessary code as there were
no failing tests upon its removal. This commit adds the necessary
regression tests for the failing condition identified in #34024
alongside the original tests added in the PR for which
WhereNode.is_summary was introduced.
2022-09-23 15:15:03 +01:00
Mariusz Felisiak ce6230aa97
Fixed #34015 -- Allowed filtering by transforms on relation fields. 2022-09-22 00:17:04 +02:00
Allen Jonathan David 10178197d5 Fixed #33966 -- Added support for using KeyTextTransform from lookup. 2022-09-16 05:36:57 +02:00
Simon Charette 3ba7f2e906
Refs #28333 -- Explicitly ordered outer qualify query on window filtering.
While most backends will propagate derived table ordering as long as
the outer query doesn't perform additional processing the SQL specs
doesn't explicitly state the ordering must be maintained.
2022-09-15 19:27:32 +02:00
David Sanders e14d08cd89 Fixed #33996 -- Fixed CheckConstraint validation on NULL values.
Bug in 667105877e.

Thanks James Beith for the report.
2022-09-13 12:48:31 +02:00
Sergey Fursov b731e88415 Fixed #31335 -- Fixed removing composed composed Meta constraints/indexes on foreign keys on MySQL. 2022-09-13 10:38:57 +02:00
Sergey Fursov ec13e801b8 Refs #31335 -- Added SchemaEditor._create_missing_fk_index() on MySQL. 2022-09-12 08:52:18 +02:00
DevilsAutumn 32797e7fbf Fixed #33975 -- Fixed __in lookup when rhs is a queryset with annotate() and alias().
This fixes clearing selected fields.
2022-09-09 08:37:46 +02:00
Simon Charette 32536b1324 Fixed #33992 -- Fixed queryset crash when aggregating over a group containing Exists.
A more in-depth solution is likely to make sure that we always GROUP BY
selected annotations or revisit how we use Query.exists() in the Exists
expression but that requires extra work that isn't suitable for a
backport.

Regression in e5a92d400a.

Thanks Fernando Flores Villaça for the report.
2022-09-08 05:50:02 +02:00
James Beith 19e838daa8 Fixed #33982 -- Fixed migrations crash when adding model with ExclusionConstraint.
Regression in 0e656c02fe.
2022-09-07 08:40:56 +02:00
DevilsAutumn 3a084831e2 Refs #33616 -- Updated BaseDatabaseWrapper.run_on_commit comment. 2022-09-06 20:50:57 +02:00
SirAbhi13 4a1150b41d Fixed #33616 -- Allowed registering callbacks that can fail in transaction.on_commit().
Thanks David Wobrock and Mariusz Felisiak for reviews.
2022-09-06 12:21:36 +02:00
Mariusz Felisiak 69fa2e8eb2
Refs #26780 -- Made prefetch_related() don't use window expressions fo sliced queries if not supported. 2022-09-06 05:54:35 +02:00
Allen Jonathan David cd1afd553f Fixed #29799 -- Allowed registering lookups per field instances.
Thanks Simon Charette and Mariusz Felisiak for reviews and mentoring
this Google Summer of Code 2022 project.
2022-09-02 10:02:24 +02:00
Simon Charette b3db6c8dcb Fixed #21204 -- Tracked field deferrals by field instead of models.
This ensures field deferral works properly when a model is involved
more than once in the same query with a distinct deferral mask.
2022-08-30 08:43:53 +02:00
Simon Charette 60613ef516 Merged nested if statements in select_related_descend(). 2022-08-30 06:59:55 +02:00
Mariusz Felisiak 4483a9b12f
Fixed #33960 -- Fixed migrations crash on SQLite < 3.26.
Regression in 0b95a96ee1.

Thanks Aristotelis Mikropoulos for the report.
2022-08-29 09:55:45 +02:00
Mohamed Karam b92ffebb0c Fixed #33954 -- Prevented models.DecimalField from accepting NaN, Inf, and -Inf values. 2022-08-27 15:51:08 +02:00
David Wobrock 806e9e2d0d Fixed #33952 -- Reallowed creating reverse foreign key managers on unsaved instances.
Thanks Claude Paroz for the report.

Regression in 7ba6ebe914.
2022-08-27 15:03:48 +02:00
Simon Charette 0701bb8e1f Fixed #33928 -- Avoided unnecessary queries when cascade updating.
Models that use SET, SET_NULL, and SET_DEFAULT as on_delete handler
don't have to fetch objects for the sole purpose of passing them back to
a follow up UPDATE query filtered by the retrieved objects primary key.

This was achieved by flagging SET handlers as _lazy_ and having the
collector logic defer object collections until the last minute. This
should ensure that the rare cases where custom on_delete handlers are
defined remain uncalled when when dealing with an empty collection of
instances.

This reduces the number queries required to apply SET handlers from
2 to 1 where the remaining UPDATE use the same predicate as the non
removed SELECT query.

In a lot of ways this is similar to the fast-delete optimization that
was added in #18676 but for updates this time. The conditions only
happen to be simpler in this case because SET handlers are always
terminal. They never cascade to more deletes that can be combined.

Thanks Renan GEHAN for the report.
2022-08-27 14:25:55 +02:00
Simon Charette a9be1dc551 Refs #33928 -- Removed unnecessary attribute assignment on on-delete updates.
Model instances retrieved for bulk field update purposes are not exposed
to the outside world and thus are not required to be kept update to
date.
2022-08-27 14:25:50 +02:00
Iuri de Silvio 166a3b3263 Fixed #33953 -- Reverted "Fixed #33201 -- Made RenameModel operation a noop for models with db_table."
Regression in afeafd6036.
This reverts afeafd6036.

Thanks Timothy Thomas for the report.
2022-08-26 06:14:44 +02:00
Simon Charette 71902e0d9f Fixed #33938 -- Fixed migration crash for m2m with a through model in another app.
Regression in aa4acc164d.

Thanks bryangeplant for the report.
2022-08-25 10:03:57 +02:00
Jan Pieter Waagmeester 2480554dc4
Fixed typo in django/db/models/fields/files.py comment. 2022-08-25 09:41:33 +02:00
Willem Van Onsem 16fffc829c Fixed #33916 -- Added support for serialization of enum.Flag in migrations. 2022-08-25 06:43:48 +02:00
Mariusz Felisiak e9fd2b5724
Refs #26511 -- Fixed json.KeyTextTransform() on MySQL/MariaDB. 2022-08-18 21:02:29 +02:00
Benoît Vinot e3cb8bcb7d Fixed #33932 -- Fixed altering AutoFields to OneToOneField on PostgreSQL.
Regression in 2eea361eff.
2022-08-17 17:06:00 +02:00
Simon Charette 242499f2dc Fixed #26780 -- Added prefetch_related() support for sliced queries.
This was made possible by window function filtering support added in
f387d024fc.
2022-08-15 10:25:20 +02:00
Simon Charette f387d024fc Refs #28333 -- Added partial support for filtering against window functions.
Adds support for joint predicates against window annotations through
subquery wrapping while maintaining errors for disjointed filter
attempts.

The "qualify" wording was used to refer to predicates against window
annotations as it's the name of a specialized Snowflake extension to
SQL that is to window functions what HAVING is to aggregates.

While not complete the implementation should cover most of the common
use cases for filtering against window functions without requiring
the complex subquery pushdown and predicate re-aliasing machinery to
deal with disjointed predicates against columns, aggregates, and window
functions.

A complete disjointed filtering implementation should likely be
deferred until proper QUALIFY support lands or the ORM gains a proper
subquery pushdown interface.
2022-08-15 08:26:26 +02:00
Haolun Chai f3f9d03edf Fixed #33901 -- Skipped varchar_pattern_ops/text_pattern_ops index creation when db_collation is set. 2022-08-15 06:28:35 +02:00
Mariusz Felisiak 5c803bc070
Fixed #33919 -- Fixed adding AutoFields on PostgreSQL.
Thanks Jack Calvin Brown for the report.

Regression in 2eea361eff.
2022-08-12 17:30:23 +02:00
Simon Charette 8c3046daad Refs #28333 -- Moved SQLCompiler's forced column aliasing logic to get_select().
This extends query composability possibilities when dealing with
subqueries which is necessary to implement window function filtering.
2022-08-11 12:39:54 +02:00
Simon Charette 35911078fa Replaced Expression.replace_references() with .replace_expressions().
The latter allows for more generic use cases beyond the currently
limited ones constraints validation has.

Refs #28333, #30581.
2022-08-11 09:53:10 +02:00