Commit Graph

3009 Commits

Author SHA1 Message Date
Ed Rivas 2798c937de Fixed #29538 -- Fixed crash of ordering by related fields when Meta.ordering contains expressions.
Thanks Simon Charette for the review.
2022-05-12 07:19:16 +02:00
Gagaro 667105877e Fixed #30581 -- Added support for Meta.constraints validation.
Thanks Simon Charette, Keryn Knight, and Mariusz Felisiak for reviews.
2022-05-10 11:22:23 +02:00
Gagaro 5d91dc8ee3 Refs #30581 -- Added Q.check() hook. 2022-05-04 08:47:52 +02:00
Simon Charette 1109e66990
Refs #33646 -- Reduced sync_to_async hops on async QuerySet iteration.
Until we add support for truly asynchronous database backends it's
actually detrimental to have complete set retrieval require multiple
hops between sync and async emulated contexts via asgiref.

By defaulting to sending the whole sync _fetch_all() to the current
context thread pool we reduce unncessary work when dealing with large
result sets since the queryset cannot be iterated anyway before all
results are retrieved and cached.
2022-05-04 08:31:59 +02:00
Gagaro 9d04711261 Refs #30581 -- Added Q.flatten(). 2022-05-03 15:31:53 +02:00
Mariusz Felisiak 271a8e73ee Refs #33646 -- Made QuerySet.raw() async-compatible. 2022-05-02 10:52:33 +02:00
Mariusz Felisiak 77926176b2 Refs #33646 -- Added RawModelIterable. 2022-05-02 10:52:33 +02:00
David Wobrock aca9bb2a12 Fixed #33413 -- Made migrations propage collations to related fields. 2022-05-02 08:13:38 +02:00
David Wobrock 87da283338 Refs #33413 -- Added collation to CharField/TextField's db_parameters. 2022-04-29 06:30:15 +02:00
Andrew Godwin 58b27e0dbb Fixed #33646 -- Added async-compatible interface to QuerySet.
Thanks Simon Charette for reviews.

Co-authored-by: Carlton Gibson <carlton.gibson@noumenal.es>
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2022-04-26 20:25:23 +02:00
marcperrinoptel 4282fd468f Fixed #33655 -- Removed unnecessary constant from GROUP BY clause for QuerySet.exists(). 2022-04-26 06:19:18 +02:00
Hannes Ljungberg ed0a2c3238
Removed unnecessary variable in Index.create_sql().
Unnecessary since fdfb3086fc.
2022-04-22 19:07:56 +02:00
Collin Anderson 0de89b6f8d
Refs #31223 -- Added __class_getitem__() to ForeignKey. 2022-04-22 14:13:12 +02:00
sarahboyce cd4da34fc1 Fixed #33004 -- Made saving objects with unsaved GenericForeignKey raise ValueError.
This aligns to the behaviour of OneToOneField and ForeignKey fields.

Thanks Jonny Park for the initial patch.
2022-04-21 10:12:28 +02:00
Mariusz Felisiak f4f2afeb45
Refs #32226 -- Fixed JSON format of QuerySet.explain() on PostgreSQL when format is uppercased.
Follow up to aba9c2de66.
2022-04-19 08:24:24 +02:00
Mariusz Felisiak a0bd006306
Made select_for_update() don't raise TransactionManagementError on databases that don't support transactions. 2022-04-14 07:53:15 +02:00
Mariusz Felisiak fdfb3086fc Fixed DatabaseFeatures.supports_index_column_ordering and related tests with MyISAM storage engine. 2022-04-13 11:56:24 +02:00
David Smith 856b528801
Removed unnecessary tuple call in SQLInsertCompiler. 2022-04-13 08:10:52 +02:00
mgaligniana 0ad5316f22 Fixed #24296 -- Made QuerySet.exists() clear selected columns for not sliced distinct querysets. 2022-04-12 08:18:22 +02:00
Himanshu-Balasamanta 06ebaa9e28 Fixed #33626 -- Cleared cache when unregistering a lookup. 2022-04-12 06:24:02 +02:00
Mariusz Felisiak 6723a26e59 Fixed CVE-2022-28347 -- Protected QuerySet.explain(**options) against SQL injection on PostgreSQL. 2022-04-11 08:59:58 +02:00
Mariusz Felisiak 93cae5cb2f Fixed CVE-2022-28346 -- Protected QuerySet.annotate(), aggregate(), and extra() against SQL injection in column aliases.
Thanks Splunk team: Preston Elder, Jacob Davis, Jacob Moore,
Matt Hanson, David Briggs, and a security researcher: Danylo Dmytriiev
(DDV_UA) for the report.
2022-04-11 08:59:33 +02:00
Simon Charette 0b31e02487 Fixed #33618 -- Fixed MTI updates outside of primary key chain. 2022-04-07 07:54:56 +02:00
sarahboyce 65effbdb10 Fixed #33471 -- Made AlterField operation a noop when changing "choices".
This also allows customizing attributes of fields that don't affect
a column definition.
2022-04-06 13:05:57 +02:00
Luke Plant 40b8a6174f Fixed #33397 -- Corrected resolving output_field for DateField/DateTimeField/TimeField/DurationFields.
This includes refactoring of CombinedExpression._resolve_output_field()
so it no longer uses the behavior inherited from Expression of guessing
same output type if argument types match, and instead we explicitly
define the output type of all supported operations.

This also makes nonsensical operations involving dates
(e.g. date + date) raise a FieldError, and adds support for
automatically inferring output_field for cases such as:
* date - date
* date + duration
* date - duration
* time + duration
* time - time
2022-03-31 11:05:23 +02:00
Luke Plant 1efea11808 Refs #33397 -- Added register_combinable_fields(). 2022-03-31 11:02:46 +02:00
Mariusz Felisiak d7eb500338 Removed unnecessary Query.get_loaded_field_names_cb() and Query.deferred_to_data()'s callback argument. 2022-03-31 10:54:59 +02:00
Mariusz Felisiak 0a3c6fe6b2 Refs #24020 -- Removed redundant Query.get_loaded_field_names().
get_loaded_field_names() is no longer called in multiple places
(see 0c7633178f) and it's redundant
with SQLCompiler.deferred_to_columns().
2022-03-31 10:54:59 +02:00
Mariusz Felisiak 0db0a25d84
Updated select_related_descend() comment.
Outdated since 0c7633178f.
2022-03-31 08:50:25 +02:00
Mariusz Felisiak fac662f479
Fixed #33598 -- Reverted "Removed unnecessary reuse_with_filtered_relation argument from Query methods."
Thanks lind-marcus for the report.

This reverts commit 0c71e0f9cf.

Regression in 0c71e0f9cf.
2022-03-30 07:31:56 +02:00
Carlton Gibson bb61f0186d Refs #32365 -- Removed internal uses of utils.timezone.utc alias.
Remaining test case ensures that uses of the alias are mapped
canonically by the migration writer.
2022-03-24 06:29:50 +01:00
Gagaro 4bd494db42 Made BaseConstraint importable from django.db.models. 2022-03-16 16:21:10 +01:00
Gagaro 1ea7e3157d Used sets for field names for exclusion.
They are used only for containment checks.
2022-03-16 11:05:09 +01:00
Gagaro bf524d229f Refs #30581 -- Allowed sql.Query to be used without model. 2022-03-16 09:33:16 +01:00
Sage Abdullah a88fab1bca Fixed #33552 -- Fixed JSONField has key lookups with numeric keys on MariaDB, MySQL, Oracle, and SQLite. 2022-03-15 06:37:35 +01:00
Ryan Heard c6b4d62fa2 Fixed #29865 -- Added logical XOR support for Q() and querysets. 2022-03-04 12:55:37 +01:00
Keryn Knight 24cc51f8fb Refs #27624 -- Optimized Query.clone() a bit. 2022-03-03 08:38:03 +01:00
Keryn Knight 6d5709ce7d Refs #27624 -- Optimized sql.Query creation by moving immutable/singleton attributes to class attributes. 2022-03-03 08:36:29 +01:00
Albert Defler 2b6a3baebe Fixed #31486 -- Deprecated passing unsaved objects to related filters.
Co-Authored-By: Hasan Ramezani <hasan.r67@gmail.com>
2022-02-25 07:51:37 +01:00
Albert Defler 7ba6ebe914 Fixed #19580 -- Unified behavior of reverse foreign key and many-to-many relations for unsaved instances. 2022-02-22 09:16:40 +01:00
Mariusz Felisiak 1e2e1be02b
Fixed #33515 -- Prevented recreation of migration for ManyToManyField to lowercased swappable setting.
Thanks Chris Lee for the report.

Regression in 4328970780.

Refs #23916.
2022-02-16 21:09:24 +01:00
Collin Anderson 35c2474f16
Made Field.error_messages a cached property.
This speeds up field creation and reduces memory usage.
2022-02-16 20:30:04 +01:00
Damian Posener 09e499a39e Fixed #33501 -- Made order_with_respect_to respect database routers. 2022-02-09 13:31:49 +01:00
Simon Charette d35ce682e3 Fixed #33506 -- Made QuerySet.bulk_update() perform atomic writes against write database.
The lack of _for_write = True assignment in bulk_update prior to
accessing self.db resulted in the db_for_read database being used to
wrap batched UPDATEs in a transaction.

Also tweaked the batch queryset creation to also ensure they are
executed against the same database as the opened transaction under all
circumstances.

Refs #23646, #33501.
2022-02-09 11:14:50 +01:00
Mariusz Felisiak 7119f40c98 Refs #33476 -- Refactored code to strictly match 88 characters line length. 2022-02-07 20:37:05 +01:00
django-bot 9c19aff7c7 Refs #33476 -- Reformatted code with Black. 2022-02-07 20:37:05 +01:00
Mariusz Felisiak 6f185a53a2
Refs #33482 -- Fixed QuerySet selecting and filtering againts negated Exists() with empty queryset.
Regression in b7d1da5a62.
2022-02-07 20:34:21 +01:00
Mariusz Felisiak c5cd878382
Refs #33476 -- Refactored problematic code before reformatting by Black.
In these cases Black produces unexpected results, e.g.

def make_random_password(
    self,
    length=10,
    allowed_chars='abcdefghjkmnpqrstuvwxyz' 'ABCDEFGHJKLMNPQRSTUVWXYZ' '23456789',
):

or

cursor.execute("""
SELECT ...
""",
    [table name],
)
2022-02-03 11:20:46 +01:00
Simon Charette b7d1da5a62 Fixed #33482 -- Fixed QuerySet filtering againts negated Exists() with empty queryset.
Thanks Tobias Bengfort for the report.
2022-02-02 07:54:19 +01:00
Mariusz Felisiak 71e7c8e737
Fixed #33468 -- Fixed QuerySet.aggregate() after annotate() crash on aggregates with default.
Thanks Adam Johnson for the report.
2022-01-31 11:33:24 +01:00
Jörg Breitbart 0af9a5fc7d Fixed #33463 -- Fixed QuerySet.bulk_update() with F() expressions. 2022-01-27 19:03:26 +01:00
Jacob Walls edbf930287 Fixed #29984 -- Added QuerySet.iterator() support for prefetching related objects.
Co-authored-by: Raphael Kimmig <raphael.kimmig@ampad.de>
Co-authored-by: Simon Charette <charette.s@gmail.com>
2022-01-25 06:12:04 +01:00
sean_c_hsu 0f6946495a Fixed #31685 -- Added support for updating conflicts to QuerySet.bulk_create().
Thanks Florian Apolloner, Chris Jerdonek, Hannes Ljungberg, Nick Pope,
and Mariusz Felisiak for reviews.
2022-01-19 20:17:42 +01:00
Mariusz Felisiak 30a0144134
Fixed #29338 -- Allowed using combined queryset in Subquery.
Thanks Eugene Kovalev for the initial patch, Simon Charette for the
review, and Chetan Khanna for help.
2022-01-17 18:01:07 +01:00
My-Name-Is-Nabil f37face331 Fixed #33435 -- Fixed invalid SQL generatered by Subquery.as_sql(). 2022-01-17 09:00:46 +01:00
Adam Johnson fdfa97fb16 Fixed #33441 -- Restored immutability of models.Field.__hash__().
Regression in 502e75f9ed.
2022-01-14 07:00:48 +01:00
Adam Johnson 08d8bccbf1 Improved Model.__init__() properties loop.
This improves readability, accumulates unrecognized arguments raise an
exception with all of them, and avoids refetching the values.
2022-01-13 11:09:37 +01:00
mgaligniana fa235004dd Fixed #13251 -- Made pre/post_delete signals dispatch the origin. 2022-01-11 08:06:18 +01:00
Allen Jonathan David 28c98d4113 Fixed #33216 -- Simpilified deconstructed paths for some expressions. 2022-01-07 11:19:29 +01:00
Jamie Matthews f5233dce30 Fixed #32511 -- Corrected handling prefetched nested reverse relationships.
When prefetching a set of child objects related to a set of parent
objects, we usually want to populate the relationship back from the
child to the parent to avoid a query when accessing that relationship
attribute. However, there's an edge case where the child queryset
itself specifies a prefetch back to the parent. In that case, we want
to use the prefetched relationship rather than populating the reverse
relationship from the parent.
2022-01-05 09:49:05 +01:00
Keryn Knight 0ed2919814 Fixed #33406 -- Avoided creation of MaxLengthValidator(None) when resolving Value.output_field for strings.
This brings the behaviour in line with Field subclasses which append to
the validators within __init__(), like BinaryField, and prevents the
creation of a validator which incorrectly throws a TypeError, if it
were used.
2022-01-04 05:51:00 +01:00
David Wobrock 72b23c04d8 Fixed #33374 -- Fixed ExpressionWrapper annotations with full queryset. 2021-12-21 06:17:04 +01:00
Adam Johnson 1be99e4e0a
Corrected return value of ForeignKey.db_check().
Field.db_check() should return None or a SQL string.
Returning [] happened to work because it’s falsey.
2021-12-10 09:22:09 +01:00
Ömer Faruk Abacı 81739a45b5 Fixed #33319 -- Fixed crash when combining with the | operator querysets with aliases that conflict. 2021-12-08 21:16:24 +01:00
Ömer Faruk Abacı f1bfdff690 Refs #33319 -- Added comment about keys/values assertion in Query.change_aliases(). 2021-12-08 21:16:24 +01:00
Erik Cederstrand 547656c850 Refs #25265 -- Allowed customizing Query's datastructure classes. 2021-12-07 13:36:50 +01:00
Hannes Ljungberg 1eaf38fa87 Fixed #33335 -- Made model validation ignore functional unique constraints.
Regression in 3aa545281e.

Thanks Hervé Le Roy for the report.
2021-12-06 07:59:11 +01:00
Mariusz Felisiak 2c7846d992
Fixed #33333 -- Fixed setUpTestData() crash with models.BinaryField on PostgreSQL.
This makes models.BinaryField pickleable on PostgreSQL.

Regression in 3cf80d3fcf.

Thanks Adam Zimmerman for the report.
2021-12-03 11:56:22 +01:00
Simon Charette e5a92d400a Fixed #33282 -- Fixed a crash when OR'ing subquery and aggregation lookups.
As a QuerySet resolves to Query the outer column references grouping logic
should be defined on the latter and proxied from Subquery for the cases where
get_group_by_cols is called on unresolved expressions.

Thanks Antonio Terceiro for the report and initial patch.
2021-12-02 07:23:33 +01:00
Simon Charette e3bde71676 Refs #32690 -- Altered lookups Query rhs alterations during initialization.
Having it happen at the lookup creation time ensures entry points
called before the compilation phase (e.g. get_group_by_cols) don't have
to duplicate the logic in charge of altering Query instances used as
rhs.

It also has the nice effect of reducing the amount of time the
alteration logic to once as opposed to multiple times if the queryset
is compiled more than once.
2021-12-02 07:00:52 +01:00
Hannes Ljungberg ed2018037d Fixed #33322 -- Fixed loss of assigned related object when saving relation with bulk_update(). 2021-11-29 06:27:22 +01:00
arsalan.ghassemi bdcda1ca9b Fixed #33309 -- Fixed QuerySet.distinct() crash on mixed case annotation. 2021-11-23 20:43:20 +01:00
Simon Charette aec71aaa5b Fixed #33304 -- Allowed passing string expressions to Window(order_by). 2021-11-23 07:58:44 +01:00
Simon Charette e06dc4571e Refs #33304 -- Enclosed aggregate ordering logic in an expression.
This greatly simplifies the implementation of contrib.postgres'
OrderableAggMixin and allows for reuse in Window expressions.
2021-11-23 07:28:27 +01:00
Jonny Park 6fa2930573 Refs #24121 -- Added __repr__() to BaseDatabaseWrapper, JoinPromoter, and SQLCompiler. 2021-11-19 07:57:02 +01:00
Matthijs Kooijman 1a5023883b Fixed #33257 -- Fixed Case() and ExpressionWrapper() with decimal values on SQLite. 2021-11-08 18:02:56 +01:00
Keryn Knight 4f8c7fd9d9 Fixed #32980 -- Made models cache related managers. 2021-11-08 08:44:12 +01:00
Hannes Ljungberg a934d377af Fixed #33262 -- Fixed crash of conditional aggregation on Exists(). 2021-11-04 06:31:32 +01:00
Hannes Ljungberg 25157033e9 Fixed #33260 -- Fixed crash when chaining QuerySet.exists() after select_for_update(of=()). 2021-11-03 20:08:14 +01:00
Keryn Knight a697424969 Fixed #32996 -- Cached PathInfos on relations.
PathInfo values are ostensibly static over the lifetime of the object
for which they're requested, so the data can be memoized, quickly
amortising the cost over the process' duration.
2021-11-03 11:27:04 +01:00
Mariusz Felisiak 8e3b1cf098
Removed unused model's Options.installed.
Unused since f9698c4391.
2021-10-28 16:08:59 +02:00
Adam Johnson 45f48ed4f7 Made F deconstruction omit 'expressions' in the path. 2021-10-21 09:40:52 +02:00
Erik Cederstrand 14c8504a37
Refs #25265 -- Allowed Query subclasses to build filters. 2021-10-13 07:31:25 +02:00
Bernd Wechner 2116238d5f Fixed #33131 -- Improved error messages for clashing reverse accessor names. 2021-10-06 12:33:15 +02:00
Simon Charette 0f3e1a54bf
Refs #26430 -- Removed unused branch in sql.Query.get_count().
Now that sql.Query.get_aggregation() properly deals with empty result
sets summary Count() annotations cannot result in None.

Unused since 9f3cce172f.
2021-10-06 07:25:17 +02:00
Mariusz Felisiak 903aaa35e5
Fixed #33159 -- Reverted "Fixed #32970 -- Changed WhereNode.clone() to create a shallow copy of children."
This reverts commit e441847eca.

A shallow copy is not enough because querysets can be reused and
evaluated in nested nodes, which shouldn't mutate JOIN aliases.

Thanks Michal Čihař for the report.
2021-09-30 11:26:17 +02:00
David Wobrock dd1fa3a31b Fixed #33018 -- Fixed annotations with empty queryset.
Thanks Simon Charette for the review and implementation idea.
2021-09-29 20:23:29 +02:00
David Wobrock ad36a198a1 Fixed #33141 -- Renamed Expression.empty_aggregate_value to empty_result_set_value. 2021-09-29 12:58:01 +02:00
Chinmoy Chakraborty b7fd668b37 Fixed #33033 -- Prevented models.DecimalField from accepting NaN values. 2021-09-28 13:20:14 +02:00
Mariusz Felisiak 5bac1719a2
Refs #32355 -- Used @functools.lru_cache as a straight decorator. 2021-09-27 09:10:58 +02:00
Keryn Knight 06c50cee0f Fixed #33124 -- Avoided accessing the database connections when not necessary.
Follow up to bf5abf1bdc.

This also caches the __getitem__ access.
2021-09-24 12:19:39 +02:00
Hasan Ramezani f997c81472 Fixed #33127 -- Added error messages on | and & operators with combined querysets. 2021-09-24 06:59:22 +02:00
Mariusz Felisiak 3b9fe906bf
Fixed #33133 -- Fixed handling NullBooleanField in historical migrations.
Regression in d992f4e3c2.
2021-09-23 10:36:30 +02:00
Keryn Knight 5353e7c250 Refs #27624 -- Optimized Query.clone() for non-combined queries.
This avoids constructing a generator expression and a new tuple if the
Query has no combined queries.
2021-09-20 12:34:18 +02:00
Keryn Knight e441847eca Fixed #32970 -- Changed WhereNode.clone() to create a shallow copy of children. 2021-09-20 07:03:10 +02:00
Adam Johnson fc91ea1e50 Refs #27624 -- Changed Query.explain_info to namedtuple. 2021-09-17 07:15:12 +02:00
Adam Johnson 2931d847c2 Optimized Query.clone() a bit.
This removes unnecessary "if ... is None" branches, which are already
shallow-copied in the __dict__.copy() call.
2021-09-17 07:12:56 +02:00
Carlton Gibson 306607d5b9 Fixed #32365 -- Made zoneinfo the default timezone implementation.
Thanks to Adam Johnson, Aymeric Augustin, David Smith, Mariusz Felisiak, Nick
Pope, and Paul Ganssle for reviews.
2021-09-16 12:11:05 +02:00
Hasan Ramezani 0a28b42b15 Fixed #33084 -- Removed incorrect system check for ManyToManyField with limit_choices_to. 2021-09-09 12:07:59 +02:00
David Wobrock 691486a5cf Fixed #33073 -- Fixed queryset crash with aggregation and empty/extra queryset annotation. 2021-09-01 20:59:16 +02:00