Commit Graph

2898 Commits

Author SHA1 Message Date
Nick Pope c35b81b864 Fixed #32951 -- Removed Query.where_class & co.
Unused since 3caf957ed5.
2021-07-22 08:49:20 +02:00
David Wobrock fd999318ad Fixed #32704 -- Fixed list of deferred fields when chaining QuerySet.defer() after only(). 2021-07-20 12:15:29 +02:00
Keryn Knight 6a970a8b46
Fixed #32944 -- Avoided unnecessary WhereNode.add() calls.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-20 11:59:21 +02:00
Nick Pope 501a8db465 Fixed #10929 -- Added default argument to aggregates.
Thanks to Simon Charette and Adam Johnson for the reviews.
2021-07-19 13:04:27 +02:00
Nick Pope 304f6ff46a
Prevented SQLCompiler.execute_sql() from closing cursor twice.
cursor_iter() helper calls cursor.close() in a finally block.
2021-07-16 13:11:46 +02:00
Daniyal f479df7f8d Refs #32508 -- Raised Type/ValueError instead of using "assert" in django.db.models.
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-15 11:43:33 +02:00
Mariusz Felisiak 0250340e37
Refs #32074 -- Used Enum.repr() format proposed for Python 3.10.
The Python's Steering Council decided to revert changes in the Enum
module (see https://bugs.python.org/issue44559) and moved them to
Python 3.11.

Follow up to 5d9b065d3f.

Thanks Nick Pope for the review.
2021-07-12 11:46:37 +02:00
Ian Foote f42ccdd835 Fixed #27021 -- Allowed lookup expressions in annotations, aggregations, and QuerySet.filter().
Thanks Hannes Ljungberg and Simon Charette for reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-09 11:43:06 +02:00
Keryn Knight 5013798fe9
Fixed #32840 -- Optimized Field.get_col().
get_col() used "self" as "output_field" when it was not given, and
unnecessarily compared "self" to "self".

Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2021-07-07 20:32:57 +02:00
Wu Haotian aba9c2de66 Fixed #32226 -- Fixed JSON format of QuerySet.explain() on PostgreSQL. 2021-07-05 09:08:39 +02:00
Simon Charette 9f3cce172f Refs #26430 -- Re-introduced empty aggregation optimization.
The introduction of the Expression.empty_aggregate_value interface
allows the compilation stage to enable the EmptyResultSet optimization
if all the aggregates expressions implement it.

This also removes unnecessary RegrCount/Count.convert_value() methods.
Disabling the empty result set aggregation optimization when it wasn't
appropriate prevented None returned for a Count aggregation value.

Thanks Nick Pope for the review.
2021-07-02 07:25:42 +02:00
Simon Charette f3112fde98 Fixed #26430 -- Fixed coalesced aggregation of empty result sets.
Disable the EmptyResultSet optimization when performing aggregation as
it might interfere with coalescence.
2021-07-02 06:40:54 +02:00
Hannes Ljungberg d8c90d4c22 Fixed #32786 -- Moved subquery ordering clearing optimization to the _in lookup.
Co-Authored-By: Simon Charette <charette.s@gmail.com>
2021-06-30 10:08:55 +02:00
Hannes Ljungberg 053141d31f Refs #32786 -- Made Query.clear_ordering() not to cause side effects by default. 2021-06-30 08:52:27 +02:00
abhiabhi94 cd124295d8 Fixed #32381 -- Made QuerySet.bulk_update() return the number of objects updated.
Co-authored-by: Diego Lima <diego.lima@lais.huol.ufrn.br>
2021-06-29 06:58:46 +02:00
Hasan Ramezani 7a9745fed4 Fixed #32863 -- Skipped system check for specifying type of auto-created primary keys on models with invalid app_label.
Regression in b5e12d490a.

Thanks Iuri de Silvio for the report.
2021-06-22 20:47:15 +02:00
Mariusz Felisiak e07609a0d1 Refs #32858, Refs #32392 -- Restored using :: shortcut syntax in Cast() on PostgreSQL.
This partly reverts commit fdfbc66331
unnecessary since b69b0c3fe8.
2021-06-22 06:28:42 +02:00
Carlton Gibson 225d96533a Fixed #30427, Fixed #16176 -- Corrected setting descriptor in Field.contribute_to_class().
Co-authored-by: Jarek Glowacki <jarekwg@gmail.com>
2021-06-15 12:01:28 +02:00
Takayuki Hirayama 0393b9262d Fixed #32812 -- Restored immutability of named values from QuerySet.values_list().
Regression in 981a072dd4.

Thanks pirelle for the report.
2021-06-04 07:23:16 +02:00
David Wobrock b9df2b74b9 Fixed #32676 -- Prevented migrations from rendering related field attributes when not passed during initialization.
Thanks Simon Charette for the implementation idea.
2021-05-28 20:25:59 +02:00
Hannes Ljungberg b746596f5f Refs #32779 -- Changed DatabaseSchemaEditor._unique_sql()/_create_unique_sql() to take fields as second parameter. 2021-05-28 10:50:27 +02:00
saeedblanchette d3d95d645f Refs #24121 -- Added __repr__() to Lookup. 2021-05-24 07:32:25 +02:00
Mariusz Felisiak f0a9413bd2 Refs #24121 -- Improved Value.__repr__(). 2021-05-24 07:26:53 +02:00
Rust Saiargaliev a24fed399c Fixed #32733 -- Skipped system check for specifying type of auto-created primary keys on abstract models.
Regression in b5e12d490a.
2021-05-18 13:02:33 +02:00
Artur Beltsov 3954bf50fb Fixed #32750 -- Fixed crash of Extract() transform on OuterRef() expressions.
Thanks Simon Charette for the review.
2021-05-17 17:51:39 +02:00
Mariusz Felisiak b55699968f
Fixed #32718 -- Relaxed file name validation in FileField.
- Validate filename returned by FileField.upload_to() not a filename
  passed to the FileField.generate_filename() (upload_to() may
  completely ignored passed filename).
- Allow relative paths (without dot segments) in the generated filename.

Thanks to Jakub Kleň for the report and review.
Thanks to all folks for checking this patch on existing projects.
Thanks Florian Apolloner and Markus Holtermann for the discussion and
implementation idea.

Regression in 0b79eb3691.
2021-05-13 08:53:44 +02:00
Simon Charette 136ff592ad Fixed #32690 -- Fixed __in lookup crash when combining with filtered aggregates.
Having lookups group by subquery right-hand-sides is likely unnecessary
in the first place but relatively large amount of work would be needed
to achieve that such as making Lookup instances proper resolvable
expressions.

Regression in 3543129822.

Thanks James A. Munsch for the report.
2021-05-05 11:41:35 +02:00
Simon Charette 96f55ccf79 Fixed #32714 -- Prevented recreation of migration for Meta.ordering with OrderBy expressions.
Regression in c8b6594305.

Thanks Kevin Marsh for the report.
2021-05-05 08:43:57 +02:00
Florian Apolloner 0b79eb3691 Fixed CVE-2021-31542 -- Tightened path & file name sanitation in file uploads. 2021-05-04 08:44:42 +02:00
Hasan Ramezani 8de4ca74ba Fixed #32693 -- Quoted and lowercased generated column aliases. 2021-05-04 07:36:56 +02:00
Nick Pope 0c71e0f9cf Removed unnecessary reuse_with_filtered_relation argument from Query methods.
In Query.join() the argument reuse_with_filtered_relation was used to
determine whether to use == or .equals(). As this area of code is
related to aliases, we only expect an instance of Join or BaseTable to
be provided - the only two classes that provide .equals().

In both cases, the implementations of __eq__() and equals() are based
on use of the "identity" property. __eq__() performs an isinstance()
check first, returning NotImplemented if required. BaseTable.equals()
then does a straightforward equality check on "identity". Join.equals()
is a little bit different as it skips checking the last element of the
"identity" property: filtered_relation. This was only included
previously when the with_filtered_relation argument was True, impossible
since bbf141bcdc.
2021-04-28 21:19:42 +02:00
Nick Pope a6c79f4987 Removed unused with_filtered_relation argument from .equals()
Unused since bbf141bcdc.
2021-04-28 20:42:20 +02:00
Simon Charette c8b6594305 Fixed #32632, Fixed #32657 -- Removed flawed support for Subquery deconstruction.
Subquery deconstruction support required implementing complex and
expensive equality rules for sql.Query objects for little benefit as
the latter cannot themselves be made deconstructible to their reference
to model classes.

Making Expression @deconstructible and not BaseExpression allows
interested parties to conform to the "expression" API even if they are
not deconstructible as it's only a requirement for expressions allowed
in Model fields and meta options (e.g. constraints, indexes).

Thanks Phillip Cutter for the report.

This also fixes a performance regression in bbf141bcdc.
2021-04-28 12:13:55 +02:00
Mariusz Felisiak 4074f38e1d Refs #32682 -- Fixed QuerySet.delete() crash on querysets with self-referential subqueries on MySQL. 2021-04-27 10:34:47 +02:00
Mariusz Felisiak 0aacbdcf27
Refs #32673 -- Fixed lookups crash when comparing against lookups on Oracle.
Follow up to 170b006ce8.
2021-04-23 16:32:53 +02:00
Simon Charette 170b006ce8 Fixed #32673 -- Fixed lookups crash when comparing against lookups on PostgreSQL.
Regression in 3a505c70e7.

Nonlitteral right-hand-sides of lookups need to be wrapped in
parentheses to avoid operator precedence ambiguities.

Thanks Charles Lirsac for the detailed report.
2021-04-23 15:38:32 +02:00
Chris Jerdonek 0461b7a6b6 Fixed #32662 -- Refactored a generator out of SQLCompiler.get_order_by().
This also renames the `asc` variable to `default_order`, markes the
`desc` variable as unused, fixes a typo in SQLCompiler.get_order_by()
docstring, and reorders some blocks in SQLCompiler._order_by_pairs().
2021-04-21 12:35:49 +02:00
Simon Charette 6d0cbe42c3 Fixed #32650 -- Fixed handling subquery aliasing on queryset combination.
This issue started manifesting itself when nesting a combined subquery
relying on exclude() since 8593e162c9 but
sql.Query.combine never properly handled subqueries outer refs in the
first place, see QuerySetBitwiseOperationTests.test_subquery_aliases()
(refs #27149).

Thanks Raffaele Salmaso for the report.
2021-04-21 09:49:15 +02:00
Tobias Bengfort 54e94640ac Refs #25287 -- Added support for multiplying and dividing DurationField by scalar values on SQLite. 2021-04-20 11:44:41 +02:00
David Beitey 4bb27c8719
Removed unnecessary line in OrderBy.as_sql().
Unnecessary since 7286eaf681.
2021-04-20 09:47:11 +02:00
Hasan Ramezani a77c9a4229 Fixed #32635 -- Fixed system check crash for reverse o2o relations in CheckConstraint.check and UniqueConstraint.condition.
Regression in b7b7df5fbc.

Thanks Szymon Zmilczak for the report.
2021-04-14 10:06:18 +02:00
Daniel Hahler 59552bea57 Corrected typo in Field.deconstruct docstring. 2021-04-13 11:44:44 +02:00
Iuri de Silvio 9760e262f8 Fixed #32627 -- Fixed QuerySet.values()/values_list() crash on combined querysets ordered by unannotated columns. 2021-04-12 21:11:40 +02:00
Adam Johnson 45a58c31e6 Fixed #32620 -- Allowed subclasses of Big/SmallAutoField for DEFAULT_AUTO_FIELD. 2021-04-08 13:17:08 +02:00
Adam Johnson d9de74141e Fixed #32442 -- Used converters on returning fields from INSERT statements. 2021-04-07 17:10:00 +02:00
Mariusz Felisiak c6859f1a68
Refs #32074 -- Backported Enum.__repr__() from Python 3.10.
Enum.__repr__() was changed in [1], we should use the same format in
Python < 3.10.

[1] https://bugs.python.org/issue40066
2021-04-07 10:28:40 +02:00
Nick Pope 2f13c476ab Fixed #31487 -- Added precision argument to Round(). 2021-03-29 09:43:08 +02:00
Hasan Ramezani ed0cc52dc3 Fixed #32585 -- Fixed Value() crash with DecimalField on SQLite. 2021-03-29 06:22:36 +02:00
Mariusz Felisiak 94463aa861
Removed trailing whitespaces in error messages. 2021-03-24 11:44:01 +01:00
Nick Pope a96c730431 Fixed #32460 -- Allowed "label"/"do_not_call_in_templates" members in model choice enums. 2021-03-24 07:45:33 +01:00
Florian Demmer 3a185cee2a Fixed #32573 -- Fixed bounds in __iso_year lookup optimization. 2021-03-23 21:27:55 +01:00
Mariusz Felisiak 71ec102b01 Fixed #32483 -- Fixed QuerySet.values()/values_list() on JSONField key transforms with booleans on SQLite.
Thanks Matthew Cornell for the report.
2021-03-23 08:28:47 +01:00
Jonathan Richards 00b0786de5 Fixed #32548 -- Fixed crash when combining Q() objects with boolean expressions. 2021-03-17 21:53:39 +01:00
Yuri Konotopov 330bc402a8 Fixed #32546 -- Avoided Meta.ordering columns in GROUP BY clauses.
Follow up to 0ddb4ebf7b.
2021-03-16 07:58:38 +01:00
Mariusz Felisiak ba9a2b7544
Refs #32508 -- Raised TypeError instead of using "assert" on unsupported operations for sliced querysets. 2021-03-10 09:16:28 +01:00
Johan Schiff d01709aae2 Fixed #24141 -- Added QuerySet.contains(). 2021-03-06 20:40:29 +01:00
Jacob Walls 73b1b225ce Fixed #22640 -- Raised TypeError when instantiating model with keyword and positional args for the same field. 2021-03-01 20:26:39 +01:00
Simon Charette 277eea8fcc Fixed #32478 -- Included nested columns referenced by subqueries in GROUP BY on aggregations.
Regression in fb3f034f1c.

Refs #31094, #31150.

Thanks Igor Pejic for the report.
2021-02-24 09:01:36 +01:00
Hannes Ljungberg 3aa545281e Fixed #30916 -- Added support for functional unique constraints.
Thanks Ian Foote and Mariusz Felisiak for reviews.
2021-02-23 20:19:53 +01:00
Hannes Ljungberg 87acbf0631 Fixed #32458 -- Made __repr__() for Index and BaseConstraint subclasses more consistent. 2021-02-19 20:25:11 +01:00
Hasan Ramezani f2bef2b7bc Fixed #32455 -- Allowed right combining Q() with boolean expressions. 2021-02-18 22:20:36 +01:00
starryrbs 466920f6d7 Fixed #32450 -- Fixed crash when ANDing/ORing an empty Q() with not pickleable Q().
Regression in bb0b6e5263.
2021-02-18 21:13:24 +01:00
Egidijus Macijauskas 6307c3f1a1 Fixed #32433 -- Added error message on QuerySet.delete() following distinct(). 2021-02-11 08:33:30 +01:00
Hasan Ramezani 63d239db03 Fixed #32411 -- Fixed __icontains lookup for JSONField on MySQL. 2021-02-05 16:14:12 +01:00
Mariusz Felisiak 5d9b065d3f Refs #32074 -- Fixed TextChoices/IntegerChoices crash on Python 3.10.
EnumMeta has a new keyword argument 'boundary' in Python 3.10. This
is a new mechanism that controls how out-of-range / invalid bits are
handled, see https://bugs.python.org/issue38250.
2021-02-04 10:47:45 +01:00
Hasan Ramezani 7cba92ec55 Fixed #32332 -- Fixed loss of parent with non-numeric pk when saving child after parent.
Follow up to 519016e5f2.
2021-02-04 06:06:00 +01:00
Tilman Koschnick fdfbc66331 Fixed #32392 -- Fixed ExclusionConstraint crash with Cast() in expressions. 2021-01-29 11:05:00 +01:00
Mariusz Felisiak a03a36121d Fixed #32367 -- Fixed system check for specifying type of auto-created primary keys for inherited PKs.
Regression in b5e12d490a.

Thanks אורי for the report.
2021-01-21 09:03:07 +01:00
Mariusz Felisiak d992f4e3c2 Refs #31369 -- Removed models.NullBooleanField per deprecation timeline. 2021-01-14 17:50:04 +01:00
Mariusz Felisiak 06eec31970 Refs #7098 -- Removed support for passing raw column aliases to order_by().
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Mariusz Felisiak 68e3ca13d7 Refs #30988 -- Removed InvalidQuery exception per deprecation timeline. 2021-01-14 17:50:04 +01:00
Mariusz Felisiak 396da8b94c Refs #30841 -- Made isnull lookup raise ValueError for non-boolean values.
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Mariusz Felisiak 5e33ec80d1 Refs #30158 -- Made alias argument required in signature of Expression.get_group_by_cols() subclasses.
Per deprecation timeline.
2021-01-14 17:50:04 +01:00
Hasan Ramezani f750377318 Fixed #32321 -- Added system checks for invalid model field names in functional indexes. 2021-01-14 12:27:00 +01:00
Hannes Ljungberg 83fcfc9ec8 Fixed #26167 -- Added support for functional indexes.
Thanks Simon Charette, Mads Jensen, and Mariusz Felisiak for reviews.

Co-authored-by: Markus Holtermann <info@markusholtermann.eu>
2021-01-13 11:47:50 +01:00
Hannes Ljungberg 4c62cdaa10 Refs #26709 -- Made Index raise ValueError on non-string fields. 2021-01-12 20:22:44 +01:00
Alexander Lyabah 415f50298f Fixed #32231 -- Allowed passing None params to QuerySet.raw(). 2021-01-05 10:37:11 +01:00
manav014 a9a7421ab8 Fixed #32294 -- Prevented ManyToManyField's hidden related name collisions between apps. 2020-12-30 13:11:47 +01:00
Mariusz Felisiak 45519937e5 Added app labels to related fields checks messages E302-E305. 2020-12-29 09:01:35 +01:00
Mariusz Felisiak ce30e750e6
Used model's Options.label where applicable.
Follow up to b7a3a6c9ef.
2020-12-29 08:56:39 +01:00
Hannes Ljungberg d23cb83c99 Refs #26167 -- Made DatabaseSchemaEditor._create_index_sql()'s fields argument optional and kwarg-only. 2020-12-23 11:16:13 +01:00
Alexey Nigin b9ba85a7ce Fixed #32089 -- Fixed prefetch_related_objects() when some objects are already fetched.
Thanks Dennis Kliban for the report and Adam Johnson for the initial
patch.

Co-authored-by: Adam Johnson <me@adamj.eu>
2020-12-16 10:52:59 +01:00
Tom Forbes b5e12d490a Fixed #31007 -- Allowed specifying type of auto-created primary keys.
This also changes the default type of auto-created primary keys
for new apps and projects to BigAutoField.
2020-12-15 11:25:46 +01:00
sage 8d7085e0fd Fixed #32252 -- Fixed __isnull=True on key transforms on SQLite and Oracle.
__isnull=True on key transforms should not match keys with NULL values.
2020-12-11 09:15:42 +01:00
sage e23484484c
Removed lhs_only argument from KeyTransform.preprocess_lhs(). 2020-12-07 10:14:27 +01:00
Artur Beltsov 48b4bae983 Fixed #32179 -- Added JSONObject database function. 2020-12-02 09:30:50 +01:00
Ian Foote 8b040e3cbb Fixed #25534, Fixed #31639 -- Added support for transform references in expressions.
Thanks Mariusz Felisiak and Simon Charette for reviews.
2020-11-27 20:42:04 +01:00
sage fe6e582421 Fixed #32203 -- Fixed QuerySet.values()/values_list() crash on key transforms with non-string values on SQLite.
Thanks Gordon Wrigley for the report.
2020-11-25 14:51:35 +01:00
Mariusz Felisiak b996424199 Simplified JSONField key transforms by using JSON_OBJECT() on Oracle. 2020-11-24 18:11:19 +01:00
Hasan Ramezani fe9c7ded29 Fixed #32200 -- Fixed grouping by ExpressionWrapper() with Q objects.
Thanks Gordon Wrigley for the report.

Regression in df32fd42b8.
2020-11-19 21:00:04 +01:00
David-Wobrock ba42569d5c Fixed #31507 -- Added QuerySet.exists() optimizations to compound queries. 2020-11-16 20:29:13 +01:00
David-Wobrock 464a4c0c59 Fixed #31496 -- Fixed QuerySet.values()/values_list() crash on combined querysets ordered by annotations. 2020-11-14 10:22:09 +01:00
Hannes Ljungberg 0773837e15 Fixed #32182 -- Fixed crash of JSONField nested key transforms with subquery annotations on PostgreSQL. 2020-11-10 07:09:58 +01:00
Simon Charette c2d4926702 Fixed #31910 -- Fixed crash of GIS aggregations over subqueries.
Regression was introduced by fff5186 but was due a long standing issue.

AggregateQuery was abusing Query.subquery: bool by stashing its
compiled inner query's SQL for later use in its compiler which made
select_format checks for Query.subquery wrongly assume the provide
query was a subquery.

This patch prevents that from happening by using a dedicated
inner_query attribute which is compiled at a later time by
SQLAggregateCompiler.

Moving the inner query's compilation to SQLAggregateCompiler.compile
had the side effect of addressing a long standing issue with
aggregation subquery pushdown which prevented converters from being
run. This is now fixed as the aggregation_regress adjustments
demonstrate.

Refs #25367.

Thanks Eran Keydar for the report.
2020-11-04 09:54:58 +01:00
David-Wobrock cfc7cd6513 Fixed #32132 -- Fixed column types in m2m intermediary tables for Positive(Big/Small)IntegerFields. 2020-11-02 20:11:58 +01:00
Christian Klus 4ac2d4fa42 Fixed #32152 -- Fixed grouping by subquery aliases.
Regression in 42c08ee465.

Thanks Simon Charette for the review.
2020-10-29 09:56:09 +01:00
Martin Thoma 302caa40e4 Made small readability improvements. 2020-10-28 20:20:20 +01:00
Simon Charette 8593e162c9 Fixed #32143 -- Used EXISTS to exclude multi-valued relationships.
As mentioned in the pre-existing split_exclude() docstring EXISTS is
easier to optimize for query planers and circumvents the IN (NULL)
handling issue.
2020-10-28 07:22:00 +01:00
Simon Charette bbf141bcdc Refs #27149 -- Fixed sql.Query identity.
By making Query subclass BaseExpression in
3543129822 the former defined it's
identity based off _construct_args which is not appropriate.
2020-10-28 07:21:53 +01:00
Nguyen You 0b4fe82c74
Improved naming consistency in BaseManage.contribute_to_class().
Matches signatures of other contribute_to_class() methods.
2020-10-27 10:51:03 +01:00