Commit Graph

5018 Commits

Author SHA1 Message Date
Mariusz Felisiak 903aaa35e5
Fixed #33159 -- Reverted "Fixed #32970 -- Changed WhereNode.clone() to create a shallow copy of children."
This reverts commit e441847eca.

A shallow copy is not enough because querysets can be reused and
evaluated in nested nodes, which shouldn't mutate JOIN aliases.

Thanks Michal Čihař for the report.
2021-09-30 11:26:17 +02:00
David Wobrock dd1fa3a31b Fixed #33018 -- Fixed annotations with empty queryset.
Thanks Simon Charette for the review and implementation idea.
2021-09-29 20:23:29 +02:00
David Wobrock ad36a198a1 Fixed #33141 -- Renamed Expression.empty_aggregate_value to empty_result_set_value. 2021-09-29 12:58:01 +02:00
Chinmoy Chakraborty b7fd668b37 Fixed #33033 -- Prevented models.DecimalField from accepting NaN values. 2021-09-28 13:20:14 +02:00
Jordan Bae f5a3243911 Fixed #33125 -- Avoided redundant unique constraint when converting a non-unique field to primary key on MySQL and PostgreSQL. 2021-09-28 06:20:37 +02:00
Mariusz Felisiak 5bac1719a2
Refs #32355 -- Used @functools.lru_cache as a straight decorator. 2021-09-27 09:10:58 +02:00
Keryn Knight 06c50cee0f Fixed #33124 -- Avoided accessing the database connections when not necessary.
Follow up to bf5abf1bdc.

This also caches the __getitem__ access.
2021-09-24 12:19:39 +02:00
Hasan Ramezani f997c81472 Fixed #33127 -- Added error messages on | and & operators with combined querysets. 2021-09-24 06:59:22 +02:00
Mariusz Felisiak 3b9fe906bf
Fixed #33133 -- Fixed handling NullBooleanField in historical migrations.
Regression in d992f4e3c2.
2021-09-23 10:36:30 +02:00
Mariusz Felisiak 221b2f85fe
Fixed #33129 -- Dropped support for MariaDB 10.2. 2021-09-22 11:57:54 +02:00
Keryn Knight 5353e7c250 Refs #27624 -- Optimized Query.clone() for non-combined queries.
This avoids constructing a generator expression and a new tuple if the
Query has no combined queries.
2021-09-20 12:34:18 +02:00
Keryn Knight e441847eca Fixed #32970 -- Changed WhereNode.clone() to create a shallow copy of children. 2021-09-20 07:03:10 +02:00
Adam Johnson fc91ea1e50 Refs #27624 -- Changed Query.explain_info to namedtuple. 2021-09-17 07:15:12 +02:00
Adam Johnson 2931d847c2 Optimized Query.clone() a bit.
This removes unnecessary "if ... is None" branches, which are already
shallow-copied in the __dict__.copy() call.
2021-09-17 07:12:56 +02:00
Carlton Gibson 306607d5b9 Fixed #32365 -- Made zoneinfo the default timezone implementation.
Thanks to Adam Johnson, Aymeric Augustin, David Smith, Mariusz Felisiak, Nick
Pope, and Paul Ganssle for reviews.
2021-09-16 12:11:05 +02:00
Hasan Ramezani 0a28b42b15 Fixed #33084 -- Removed incorrect system check for ManyToManyField with limit_choices_to. 2021-09-09 12:07:59 +02:00
David Wobrock 691486a5cf Fixed #33073 -- Fixed queryset crash with aggregation and empty/extra queryset annotation. 2021-09-01 20:59:16 +02:00
Mariusz Felisiak 338fc0e7f1 Fixed #33080 -- Preserved nullability of textual fields on Oracle.
Thanks Matt Hoskins for the report.
2021-09-01 20:54:12 +02:00
Mariusz Felisiak 1eb3f500a4
Fixed #33057 -- Fixed recreation of foreign key constraints in m2m tables when altering type of referenced primary key on Oracle. 2021-08-31 13:43:10 +02:00
Jacob Walls 3219dd3388 Fixed #24900 -- Allowed migrating backward to squashed migrations. 2021-08-30 12:08:04 +02:00
Mateo Radman 02bc7161ec Fixed #32900 -- Improved migrations questioner prompts. 2021-08-27 13:27:41 +02:00
Mariusz Felisiak fa1d7ba5b9
Refs #29898 -- Changed fields in ProjectState's relation registry to dict. 2021-08-26 07:49:37 +02:00
Manav Agarwal 196a99da5d Refs #29898 -- Made ProjectState encapsulate alterations in relations registry.
Thanks Simon Charette and Chris Jerdonek for reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-08-25 09:12:01 +02:00
Tim Graham 022d29c934
Refs #10929 -- Allowed NowUTC SQL customization for third-party backends. 2021-08-24 08:28:03 +02:00
Manav Agarwal 3e6a3e8853 Refs #29898 -- Refactored out ProjectState.resolve_model_relations()/resolve_model_field_relations() hooks. 2021-08-20 13:45:21 +02:00
Matjaz Gregoric 518ce7a51f Fixed #33017 -- Fixed storage engine introspection on MySQL.
This also improves performance on MySQL instances with a large number
of databases, since querying the information_schema table can be very
slow
2021-08-20 11:05:21 +02:00
Matjaz Gregoric e9aa20e4e1 Optimized DatabaseSchemaEditor._field_should_be_indexed() on MySQL. 2021-08-20 10:30:02 +02:00
Keryn Knight bf5abf1bdc Fixed #33025 -- Avoided accessing the database connections in Query.build_lookup() when not necessary.
Of the built-in backends, only Oracle treats empty strings and nulls as
equal, so avoid testing the default connection backend for
interprets_empty_strings_as_nulls if it can be established from the
lookup that it wouldn't affect the lookup instance returned. This
improves performance a small amount for most lookups being built,
because accessing the connections requires touching the thread critical
`Local` which is an expensive operation.
2021-08-19 13:28:59 +02:00
Chris Jerdonek 7800596924 Fixed #33014 -- Made ProjectState raise exception when real_apps argument is not a set. 2021-08-19 10:18:51 +02:00
Mariusz Felisiak 54a30a7a00 Refs #29898 -- Changed ProjectState.real_apps to set. 2021-08-11 09:01:14 +02:00
Mariusz Felisiak b64db05b9c
Removed unnecessary WhereNode.is_summary.
Unnecessary since its introduction in 1df89a60c5.
2021-08-09 19:23:29 +02:00
Keryn Knight 921e4ccb77
Fixed #33003 -- Removed **kwargs from QuerySet._chain().
The functionality of updating the __dict__ was only present to allow
for pickling a Prefetch object, which is a comparably rare operation.
Forcing the __getstate__() implementation to handle it allows the
chaining operation to be slightly faster, which is much more common.
2021-08-09 13:28:40 +02:00
Nick Touran 5d4f21b16f Fixed #32983 -- Added system check for redundant related_name on symmetrical M2M fields.
Since ManyToManyFields defined with `symmetrical=True` do not add a
related field to the target model, including a `related_name` argument
will never do what the coder likely expects. This makes including
a related_name with a symmetrical model raise a system check warning.

ticket-32983
2021-08-06 11:57:02 +02:00
Jacob Walls 202d3e193a Fixed typos in migrations tests, comments, and error message. 2021-08-04 09:28:23 +02:00
Chris Jerdonek 6f5e07a84d Refs #32966 -- Refactored out DateTimeCheckMixin._check_if_value_fixed(). 2021-07-30 10:15:43 +02:00
Chris Jerdonek 6fa5d05dba Refs #32966 -- Simplified the _check_fix_default_value() implementations. 2021-07-30 09:31:29 +02:00
Chris Jerdonek eebebfe0a9 Refs #32966 -- Added _to_naive() and _get_naive_now() for use in DateTimeCheckMixin classes. 2021-07-30 09:31:24 +02:00
Chris Jerdonek 542e749475 Fixed #32966 -- Fixed TimeField.check() crash for timezone-aware times in default when USE_TZ = True. 2021-07-30 09:15:39 +02:00
Keryn Knight 9662193aea Refs #32946 -- Changed internal usage of dynamic Q() objects construction to use non-kwargs initialization.
This prefers non-kwargs construction of dynamically generated Q()
objects to create a single Q() object instead of many and then
combining them, where possible.
2021-07-28 09:38:46 +02:00
Keryn Knight 5b8ef8aa5b Refs #32946 -- Changed Query.add_filter() to take two arguments. 2021-07-28 09:38:42 +02:00
Chris Jerdonek 5fee36973c
Optimized BaseDatabaseSchemaEditor._effective_default() a bit,
This eliminates an unneeded call to datetime.now() when
field.get_internal_type() equals "DateTimeField".
2021-07-28 08:12:01 +02:00
Chris Jerdonek fe074c96a3 Refs #32962 -- Simplified NULL logic in BaseDatabaseSchemaEditor._iter_column_sql(). 2021-07-27 20:10:25 +02:00
Chris Jerdonek 809c45ea3c Fixed #32962 -- Consolidated string concatenations in BaseDatabaseSchemaEditor.column_sql(). 2021-07-27 20:10:20 +02:00
David Smith 1cba320786
Refs #32956 -- Changed "afterwards" to "afterward" in docs and comments.
This also removes unnecessary comments with the previous spelling.

AP Stylebook has a short entry to advise the preferred spelling for
"en-us". "Afterwards" is preferred in British English.
2021-07-27 10:41:51 +02:00
Jordan Bae 3d9040a50b Refs #32743 -- Fixed recreation of foreign key constraints when altering type of referenced primary key with MTI.
Follow up to 325d7710ce.
2021-07-27 07:30:33 +02:00
David Wobrock 325d7710ce Fixed #32743 -- Added foreign key altering when altering type of referenced primary key with MTI. 2021-07-26 08:51:56 +02:00
Tom Wojcik 20226fcd46 Fixed #32947 -- Fixed hash() crash on reverse M2M relation when through_fields is a list.
Regression in c32d8f33d8.
2021-07-26 06:09:29 +02:00
Nick Pope c35b81b864 Fixed #32951 -- Removed Query.where_class & co.
Unused since 3caf957ed5.
2021-07-22 08:49:20 +02:00
David Wobrock fd999318ad Fixed #32704 -- Fixed list of deferred fields when chaining QuerySet.defer() after only(). 2021-07-20 12:15:29 +02:00
Keryn Knight 6a970a8b46
Fixed #32944 -- Avoided unnecessary WhereNode.add() calls.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-20 11:59:21 +02:00
Nick Pope 501a8db465 Fixed #10929 -- Added default argument to aggregates.
Thanks to Simon Charette and Adam Johnson for the reviews.
2021-07-19 13:04:27 +02:00
Nick Pope 304f6ff46a
Prevented SQLCompiler.execute_sql() from closing cursor twice.
cursor_iter() helper calls cursor.close() in a finally block.
2021-07-16 13:11:46 +02:00
Daniyal f479df7f8d Refs #32508 -- Raised Type/ValueError instead of using "assert" in django.db.models.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-15 11:43:33 +02:00
Mariusz Felisiak 0250340e37
Refs #32074 -- Used Enum.repr() format proposed for Python 3.10.
The Python's Steering Council decided to revert changes in the Enum
module (see https://bugs.python.org/issue44559) and moved them to
Python 3.11.

Follow up to 5d9b065d3f.

Thanks Nick Pope for the review.
2021-07-12 11:46:37 +02:00
Ian Foote f42ccdd835 Fixed #27021 -- Allowed lookup expressions in annotations, aggregations, and QuerySet.filter().
Thanks Hannes Ljungberg and Simon Charette for reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-09 11:43:06 +02:00
Mariusz Felisiak 77b88fe621
Fixed #32908 -- Allowed select_for_update(skip_locked) on MariaDB 10.6+. 2021-07-08 06:51:10 +02:00
Keryn Knight 5013798fe9
Fixed #32840 -- Optimized Field.get_col().
get_col() used "self" as "output_field" when it was not given, and
unnecessarily compared "self" to "self".

Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-07 20:32:57 +02:00
Mariusz Felisiak 355ecd1416
Fixed inspectdb and schema tests on MariaDB 10.6+.
The utf8 character set (and related collations) is by default an alias
for utf8mb3 on MariaDB 10.6+.
2021-07-07 17:02:20 +02:00
Mateo Radman 2231429991 Refs #32508 -- Raised ImproperlyConfigured/TypeError instead of using "assert". 2021-07-05 13:31:03 +02:00
Wu Haotian aba9c2de66 Fixed #32226 -- Fixed JSON format of QuerySet.explain() on PostgreSQL. 2021-07-05 09:08:39 +02:00
Nick Pope fa35c8bdbc
Fixed #30934 -- Included database alias in django.db.backends log messages.
This is useful when working with database routing as you want to know
where each query is being executed.

Co-authored-by: David Winterbottom <david.winterbottom@gmail.com>
2021-07-02 15:36:53 +02:00
Simon Charette 9f3cce172f Refs #26430 -- Re-introduced empty aggregation optimization.
The introduction of the Expression.empty_aggregate_value interface
allows the compilation stage to enable the EmptyResultSet optimization
if all the aggregates expressions implement it.

This also removes unnecessary RegrCount/Count.convert_value() methods.
Disabling the empty result set aggregation optimization when it wasn't
appropriate prevented None returned for a Count aggregation value.

Thanks Nick Pope for the review.
2021-07-02 07:25:42 +02:00
Simon Charette f3112fde98 Fixed #26430 -- Fixed coalesced aggregation of empty result sets.
Disable the EmptyResultSet optimization when performing aggregation as
it might interfere with coalescence.
2021-07-02 06:40:54 +02:00
abhiabhi94 fde6fb2898 Fixed #32893 -- Fixed serialization of models.Model class in migrations.
Migrations assumed that an import of the models.Model class must
already be included when it's serialized, but for models with only
custom fields this was not necessarily the case.

Thanks Jaap Joris Vens for the report.
2021-07-01 12:42:32 +02:00
Hannes Ljungberg d8c90d4c22 Fixed #32786 -- Moved subquery ordering clearing optimization to the _in lookup.
Co-Authored-By: Simon Charette <charette.s@gmail.com>
2021-06-30 10:08:55 +02:00
Hannes Ljungberg 053141d31f Refs #32786 -- Made Query.clear_ordering() not to cause side effects by default. 2021-06-30 08:52:27 +02:00
manav014 503ee41497 Refs #29898 -- Moved state_forwards()'s logic from migration operations to ProjectState.
Thanks Simon Charette and Markus Holtermann for reviews.
2021-06-29 12:03:30 +02:00
abhiabhi94 cd124295d8 Fixed #32381 -- Made QuerySet.bulk_update() return the number of objects updated.
Co-authored-by: Diego Lima <diego.lima@lais.huol.ufrn.br>
2021-06-29 06:58:46 +02:00
Hasan Ramezani 7a9745fed4 Fixed #32863 -- Skipped system check for specifying type of auto-created primary keys on models with invalid app_label.
Regression in b5e12d490a.

Thanks Iuri de Silvio for the report.
2021-06-22 20:47:15 +02:00
manav014 d718d99017 Refs #29898 -- Moved django.db.migrations.operations.utils to django.db.migrations.utils. 2021-06-22 09:26:18 +02:00
Mariusz Felisiak e07609a0d1 Refs #32858, Refs #32392 -- Restored using :: shortcut syntax in Cast() on PostgreSQL.
This partly reverts commit fdfbc66331
unnecessary since b69b0c3fe8.
2021-06-22 06:28:42 +02:00
Carlton Gibson 225d96533a Fixed #30427, Fixed #16176 -- Corrected setting descriptor in Field.contribute_to_class().
Co-authored-by: Jarek Glowacki <jarekwg@gmail.com>
2021-06-15 12:01:28 +02:00
Mariusz Felisiak fa0433d05f Fixed #32832 -- Fixed adding BLOB/TEXT nullable field with default on MySQL 8.0.13+.
Regression in d4ac23bee1.

Thanks Omkar Deshpande for the report.
2021-06-10 20:03:43 +02:00
Jacob Walls c0e29cec83 Fixed #25255 -- Recorded unapplied squashed migrations. 2021-06-08 08:40:34 +02:00
Takayuki Hirayama 0393b9262d Fixed #32812 -- Restored immutability of named values from QuerySet.values_list().
Regression in 981a072dd4.

Thanks pirelle for the report.
2021-06-04 07:23:16 +02:00
Mariusz Felisiak e703b152c6
Fixed #32793 -- Fixed loss of precision for temporal operations with DecimalFields on MySQL.
Regression in 1e38f1191d.

Thanks Mohsen Tamiz for the report.
2021-06-01 15:11:42 +02:00
David Sanders 5685b7cd73
Fixed typos in comments and docs. 2021-05-29 12:51:14 +02:00
David Wobrock b9df2b74b9 Fixed #32676 -- Prevented migrations from rendering related field attributes when not passed during initialization.
Thanks Simon Charette for the implementation idea.
2021-05-28 20:25:59 +02:00
Hannes Ljungberg b746596f5f Refs #32779 -- Changed DatabaseSchemaEditor._unique_sql()/_create_unique_sql() to take fields as second parameter. 2021-05-28 10:50:27 +02:00
Hannes Ljungberg 3e0fdf5546
Fixed #32780 -- Made Add/RemoveConstraint operations a noop for covering/deferrable unique constraints on SQLite. 2021-05-25 11:34:25 +02:00
saeedblanchette d3d95d645f Refs #24121 -- Added __repr__() to Lookup. 2021-05-24 07:32:25 +02:00
Mariusz Felisiak f0a9413bd2 Refs #24121 -- Improved Value.__repr__(). 2021-05-24 07:26:53 +02:00
Hannes Ljungberg 7ef2398e81 Fixed #32777 -- Passed table reference as a string to DatabaseSchemaEditor._index_columns(). 2021-05-24 06:31:48 +02:00
Yuekui Li 5e04e84d67 Fixed #32503 -- Fixed altering BLOB/TEXT field to non-nullable with default on MySQL 8.0.13+.
MySQL 8.0.13+ supports defaults for BLOB/TEXT but not in the
ALTER COLUMN statement.

Regression in 6b16c91157.

Thanks Matt Westcott for the report.
2021-05-21 13:34:37 +02:00
Rust Saiargaliev a24fed399c Fixed #32733 -- Skipped system check for specifying type of auto-created primary keys on abstract models.
Regression in b5e12d490a.
2021-05-18 13:02:33 +02:00
Artur Beltsov 3954bf50fb Fixed #32750 -- Fixed crash of Extract() transform on OuterRef() expressions.
Thanks Simon Charette for the review.
2021-05-17 17:51:39 +02:00
Mariusz Felisiak b55699968f
Fixed #32718 -- Relaxed file name validation in FileField.
- Validate filename returned by FileField.upload_to() not a filename
  passed to the FileField.generate_filename() (upload_to() may
  completely ignored passed filename).
- Allow relative paths (without dot segments) in the generated filename.

Thanks to Jakub Kleň for the report and review.
Thanks to all folks for checking this patch on existing projects.
Thanks Florian Apolloner and Markus Holtermann for the discussion and
implementation idea.

Regression in 0b79eb3691.
2021-05-13 08:53:44 +02:00
Nick Pope 1061f52436 Fixed #32732 -- Removed usage of deprecated 'db' and 'passwd' connection options in MySQL backend.
The 'db' and 'passwd' connection options have been deprecated, use
'database' and 'password' instead (available since mysqlclient >= 1.3.8).

This also allows the 'database' option in DATABASES['OPTIONS'] on MySQL.
2021-05-12 12:21:57 +02:00
Mariusz Felisiak b1a4b1f0bd Fixed #32722 -- Fixed comparing to TruncTime() on Oracle. 2021-05-11 09:19:25 +02:00
Alex Hill c4ee3b208a Fixed #32699 -- Fixed comparing to TruncTime() with 0 microseconds on MySQL. 2021-05-07 08:13:40 +02:00
Simon Charette 136ff592ad Fixed #32690 -- Fixed __in lookup crash when combining with filtered aggregates.
Having lookups group by subquery right-hand-sides is likely unnecessary
in the first place but relatively large amount of work would be needed
to achieve that such as making Lookup instances proper resolvable
expressions.

Regression in 3543129822.

Thanks James A. Munsch for the report.
2021-05-05 11:41:35 +02:00
Simon Charette 96f55ccf79 Fixed #32714 -- Prevented recreation of migration for Meta.ordering with OrderBy expressions.
Regression in c8b6594305.

Thanks Kevin Marsh for the report.
2021-05-05 08:43:57 +02:00
Florian Apolloner 0b79eb3691 Fixed CVE-2021-31542 -- Tightened path & file name sanitation in file uploads. 2021-05-04 08:44:42 +02:00
Hasan Ramezani 8de4ca74ba Fixed #32693 -- Quoted and lowercased generated column aliases. 2021-05-04 07:36:56 +02:00
Mariusz Felisiak 1f643c28b5
Fixed #32653 -- Made quoting names in the Oracle backend consistent with db_table. 2021-04-30 12:59:07 +02:00
Nick Pope 0c71e0f9cf Removed unnecessary reuse_with_filtered_relation argument from Query methods.
In Query.join() the argument reuse_with_filtered_relation was used to
determine whether to use == or .equals(). As this area of code is
related to aliases, we only expect an instance of Join or BaseTable to
be provided - the only two classes that provide .equals().

In both cases, the implementations of __eq__() and equals() are based
on use of the "identity" property. __eq__() performs an isinstance()
check first, returning NotImplemented if required. BaseTable.equals()
then does a straightforward equality check on "identity". Join.equals()
is a little bit different as it skips checking the last element of the
"identity" property: filtered_relation. This was only included
previously when the with_filtered_relation argument was True, impossible
since bbf141bcdc.
2021-04-28 21:19:42 +02:00
Nick Pope a6c79f4987 Removed unused with_filtered_relation argument from .equals()
Unused since bbf141bcdc.
2021-04-28 20:42:20 +02:00
Simon Charette c8b6594305 Fixed #32632, Fixed #32657 -- Removed flawed support for Subquery deconstruction.
Subquery deconstruction support required implementing complex and
expensive equality rules for sql.Query objects for little benefit as
the latter cannot themselves be made deconstructible to their reference
to model classes.

Making Expression @deconstructible and not BaseExpression allows
interested parties to conform to the "expression" API even if they are
not deconstructible as it's only a requirement for expressions allowed
in Model fields and meta options (e.g. constraints, indexes).

Thanks Phillip Cutter for the report.

This also fixes a performance regression in bbf141bcdc.
2021-04-28 12:13:55 +02:00
Konstantin Alekseev 6e742dabc9 Fixed #32687 -- Restored passing process’ environment to underlying tool in dbshell on PostgreSQL.
Regression in bbe6fbb876.
2021-04-27 10:43:35 +02:00
Mariusz Felisiak 4074f38e1d Refs #32682 -- Fixed QuerySet.delete() crash on querysets with self-referential subqueries on MySQL. 2021-04-27 10:34:47 +02:00