Commit Graph

768 Commits

Author SHA1 Message Date
Daniel Hillier 60626162f7 Fixed #31866 -- Fixed locking proxy models in QuerySet.select_for_update(of=()). 2020-08-11 11:55:10 +02:00
Alexandr Tatarinov f4ac167119 Fixed #27719 -- Added QuerySet.alias() to allow creating reusable aliases.
QuerySet.alias() allows creating reusable aliases for expressions that
don't need to be selected but are used for filtering, ordering, or as
a part of complex expressions.

Thanks Simon Charette for reviews.
2020-07-31 13:19:33 +02:00
matt ferrante 7d6916e827 Fixed #29789 -- Added support for nested relations to FilteredRelation. 2020-07-09 20:24:00 +02:00
Mariusz Felisiak ae8338daf3
Fixed #31767 -- Fixed QuerySet.none() on combined queryset. 2020-07-07 12:15:05 +02:00
Jacob Walls 512da9d585 Fixed #23797 -- Fixed QuerySet.exclude() when rhs is a nullable column. 2020-07-06 10:15:37 +02:00
Hasan Ramezani 037a624120 Fixed #31657 -- Fixed ordering by attnames of self-referential ForeignKeys. 2020-06-10 11:14:26 +02:00
Nicolas Baccelli b38d44229f Fixed #31664 -- Reallowed using non-expressions having filterable attribute as rhs in queryset filters.
Regression in 4edad1ddf6.
2020-06-08 08:17:23 +02:00
Laurent Tramoy 2aac176e86 Fixed #31614 -- Fixed aliases ordering by OrderBy() expressions of combined queryset. 2020-05-26 21:14:45 +02:00
Simon Charette 972000420e Fixed #31624 -- Avoided subquery usage on QuerySet.all().delete().
Thanks Adam Johnson for the report.

Regression in 7acef095d7.
2020-05-25 08:14:49 +02:00
Manuel Weitzman a4e6030904 Fixed #30375 -- Added FOR NO KEY UPDATE support to QuerySet.select_for_update() on PostgreSQL. 2020-05-21 10:51:10 +02:00
Mariusz Felisiak 3a941230c8
Fixed #31584 -- Fixed crash when chaining values()/values_list() after Exists() annotation and aggregation on Oracle.
Oracle requires the EXISTS expression to be wrapped in a CASE WHEN in
the GROUP BY clause.

Regression in efa1908f66.
2020-05-14 15:07:08 +02:00
Simon Charette adfbf653dc Fixed #31568 -- Fixed alias reference when aggregating over multiple subqueries.
691def10a0 made all Subquery() instances
equal to each other which broke aggregation subquery pushdown which
relied on object equality to determine which alias it should select.

Subquery.__eq__() will be fixed in an another commit but
Query.rewrite_cols() should haved used object identity from the start.

Refs #30727, #30188.

Thanks Makina Corpus for the report.
2020-05-14 09:31:15 +02:00
Simon Charette 42c08ee465 Fixed #31566 -- Fixed aliases crash when chaining values()/values_list() after annotate() with aggregations and subqueries.
Subquery annotation references must be resolved if they are excluded
from the GROUP BY clause by a following .values() call.

Regression in fb3f034f1c.

Thanks Makina Corpus for the report.
2020-05-14 08:16:16 +02:00
Jon Dufresne d6aff369ad Refs #30116 -- Simplified regex match group access with Match.__getitem__().
The method has been available since Python 3.6. The shorter syntax is
also marginally faster.
2020-05-11 12:01:28 +02:00
Adam Johnson 67f9d076cf
Avoided unnecessary recompilation of ordering clause regex in SQLCompiler. 2020-04-22 20:28:01 +02:00
Simon Charette 513948735b Fixed #31426 -- Added proper field validation to QuerySet.order_by().
Resolve the field reference instead of using fragile regex based string
reference validation.
2020-04-06 10:19:49 +02:00
Simon Charette 98ea4f0f46 Refs #7098 -- Deprecated passing raw column aliases to order_by().
Now that order_by() has expression support passing RawSQL() can achieve
the same result.

This was also already supported through QuerySet.extra(order_by) for
years but this API is more or less deprecated at this point.
2020-04-06 10:01:02 +02:00
Hasan Ramezani 4237050684 Fixed #31420 -- Fixed crash when filtering subquery annotation against a SimpleLazyObject.
Thanks Simon Charette for the solution and analysis.
2020-04-06 07:11:29 +02:00
Adam Johnson c06d7c9239 Refs #31403 -- Made SQLInsertCompiler.execute_sql() always return an iterable of rows.
List of tuples.
2020-03-31 07:12:11 +02:00
Hasan Ramezani 10866a10fe Fixed #31377 -- Disabled grouping by aliases on QuerySet.values()/values_list() when they collide with field names.
Regression in fb3f034f1c.

Thanks Holovashchenko Vadym for the report.
2020-03-25 09:13:46 +01:00
Jon Dufresne 013147fae2 Fixed #31285 -- Fixed inherited Meta.ordering of "-pk". 2020-03-19 12:57:07 +01:00
Abhijeet Viswa 1712a76b9d Fixed #31246 -- Fixed locking models in QuerySet.select_for_update(of=()) for related fields and parent link fields with multi-table inheritance.
Partly regression in 0107e3d105.
2020-02-11 20:43:57 +01:00
Jon Dufresne 3259983f56 Fixed #31233 -- Closed database connections and cursors after use. 2020-02-06 15:35:23 +01:00
Nick Pope 335c9c94ac Simplified imports from django.db and django.contrib.gis.db. 2020-02-04 13:20:06 +01:00
Mariusz Felisiak 6b178a3e93
Fixed #31217 -- Made QuerySet.values()/values_list() group by not selected annotations with aggregations used in order_by().
Regression in 59b4e99dd0.

Thanks Jon Dufresne for the report and Simon Charette for the review.
2020-02-03 07:48:11 +01:00
Mariusz Felisiak 59b4e99dd0
Refs #31136 -- Made QuerySet.values()/values_list() group only by selected annotation.
Regression in 0f843fdd5b.
2020-01-15 09:32:42 +01:00
Mariusz Felisiak 0f843fdd5b
Fixed #31136 -- Disabled grouping by aliases on QuerySet.values()/values_list().
Regression in fb3f034f1c.

Thanks Sigurd Ljødal for the report.
2020-01-04 20:49:11 +01:00
Simon Charette 720de4d044 Fixed #31109 -- Disabled grouping by aliases on QuerySet.exists().
Clearing the SELECT clause in Query.has_results was orphaning GROUP BY
references to it.

Thanks Thierry Bastian for the report and Baptiste Mispelon for the
bisect.

Regression in fb3f034f1c.
2019-12-23 09:24:03 +01:00
Simon Charette 5a4d7285bd Fixed #31094 -- Included columns referenced by subqueries in GROUP BY on aggregations.
Thanks Johannes Hoppe for the report.

Regression in fb3f034f1c.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2019-12-19 10:04:56 +01:00
Baptiste Mispelon 09341856ed Used Signature API instead of deprecated inspect.getcallargs().
inspect.getcallargs() was deprecated in Python 3.5 and the Signature
API (PEP 362) has better support for decorated functions (by default,
it follows the __wrapped__ attribute set by functools.wraps for
example).
2019-12-10 09:36:34 +01:00
Alex Aktsipetrov 723fc7fcf6 Made Query.check_filterable() use bool by default instead of str. 2019-12-06 13:52:36 +01:00
Alex Aktsipetrov bf12273db4 Fixed #31060 -- Reallowed window expressions to be used in conditions outside of queryset filters.
Regression in 4edad1ddf6.

Thanks utapyngo for the report.
2019-12-06 13:52:16 +01:00
Mariusz Felisiak 0107e3d105
Fixed #30953 -- Made select_for_update() lock queryset's model when using "self" with multi-table inheritance.
Thanks Abhijeet Viswa for the report and initial patch.
2019-12-02 07:57:19 +01:00
Simon Charette f97a6123c0 Refs #25367 -- Made Query.build_filter() raise TypeError on non-conditional expressions. 2019-11-21 11:56:35 +01:00
Simon Charette e9a0e1d4f6 Fixed #30484 -- Added conditional expressions support to CheckConstraint. 2019-11-21 11:56:35 +01:00
Simon Charette 37e6c5b79b Refs #25367 -- Moved conditional expression wrapping to the Exact lookup. 2019-11-21 11:56:35 +01:00
Simon Charette 306b687520 Refs #11964 -- Removed SimpleCol in favor of Query(alias_cols).
This prevent having to pass simple_col through multiple function calls
by defining whether or not references should be resolved with aliases
at the Query level.
2019-11-21 11:56:35 +01:00
Simon Charette 3ca77e2b84 Replaced QueryWrapper single usage with RawSQL. 2019-11-18 10:45:24 +01:00
George Marshall 8be79984dc Fixed #30971 -- Prevented Query.resolve_lookup_value() from coercing list values to tuples.
Regression in 8a281aa7fe.
2019-11-13 08:27:26 +01:00
Hasan Ramezani e3d0b4d550 Fixed #30899 -- Lazily compiled import time regular expressions. 2019-10-29 09:22:26 +01:00
Simon Charette 7acef095d7 Fixed #23576 -- Implemented multi-alias fast-path deletion in MySQL backend.
This required moving the entirety of DELETE SQL generation to the
compiler where it should have been in the first place and implementing
a specialized compiler on MySQL/MariaDB.

The MySQL compiler relies on the "DELETE table FROM table JOIN" syntax
for queries spanning over multiple tables.
2019-10-24 12:24:53 +02:00
Hasan Ramezani 6a75cea76a Fixed #30854 -- Fixed QuerySet.select_related() with multiple FilteredRelations. 2019-10-11 08:10:48 +02:00
Hasan Ramezani e1ae2b0050 Refs #30854 -- Moved local_setter() outside the loop in SQLCompiler.get_related_selections(). 2019-10-11 08:07:07 +02:00
Mads Jensen 129583a0d3 Removed some outdated backwards compatibility imports and misleading comments.
EmptyResultSet moved in 46509cf13d.
FieldDoesNotExist moved in 8958170755.
BoundField and pretty_name moved in 8550161e53.
EMPTY_VALUES moved in 471596fc1a.
BaseRunserverCommand moved in 5c53e30607.
2019-09-24 15:18:53 +02:00
Mariusz Felisiak 01104368ff Refs #29444 -- Removed redundant DatabaseFeatures.can_return_multiple_columns_from_insert.
Unnecessary since b31e63879e.
2019-09-24 10:37:22 +02:00
Johannes Hoppe b31e63879e Fixed #29444 -- Allowed returning multiple fields from INSERT statements on Oracle. 2019-09-24 10:22:43 +02:00
Simon Charette 37f8f29377 Fixed #30796 -- Prevented select_related() from mutating a queryset on chaining.
Thanks Darren Maki for the report.
2019-09-24 09:17:45 +02:00
Mariusz Felisiak 0ddb4ebf7b Refs #14357 -- Made Meta.ordering not affect GROUP BY queries.
Per deprecation timeline.
2019-09-10 12:01:00 +02:00
Vojtech Bocek b1d37fea8f Fixed #28107 -- Added DatabaseFeatures.allows_group_by_selected_pks_on_model() to allow enabling optimization for unmanaged models. 2019-09-09 14:04:46 +02:00
Johannes Hoppe 7254f1138d Refs #29444 -- Allowed returning multiple fields from INSERT statements on PostgreSQL.
Thanks Florian Apolloner, Tim Graham, Simon Charette, Nick Pope, and
Mariusz Felisiak for reviews.
2019-09-09 10:51:14 +02:00
Simon Charette 13a8884a08 Fixed #30739 -- Fixed exclusion of multi-valued lookup against outer rhs.
OuterRef right hand sides have to be nested, just like F rhs have to,
during the subquery pushdown split_exclude performs to ensure they are
resolved against the outer query aliases.
2019-09-02 10:52:36 +02:00
Matthew Schinckel 4137fc2efc Fixed #25367 -- Allowed boolean expressions in QuerySet.filter() and exclude().
This allows using expressions that have an output_field that is a
BooleanField to be used directly in a queryset filters, or in the
When() clauses of a Case() expression.

Thanks Josh Smeaton, Tim Graham, Simon Charette, Mariusz Felisiak, and
Adam Johnson for reviews.

Co-Authored-By: NyanKiyoshi <hello@vanille.bid>
2019-08-29 09:45:29 +02:00
Simon Charette 8b4a43dda7 Fixed #29545 -- Fixed using filter lookups againts nested subquery expressions.
Made sql.Where resolve lhs of its child nodes. This is necessary to
allow filter lookups against nested subquery expressions to properly
resolve their OuterRefs to Cols.

Thanks Oskar Persson for the simplified test case.
2019-08-17 11:24:35 +02:00
Andrew Brown 8a281aa7fe Fixed #30687 -- Fixed using of OuterRef() expressions in distance lookups. 2019-08-16 06:50:33 +02:00
Simon Charette fff5186d32 Refs #25367 -- Moved select_format hook to BaseExpression.
This will expose an intermediary hook for expressions that need special
formatting when used in a SELECT clause.
2019-08-13 06:48:14 +02:00
aaktsipetrov 4edad1ddf6 Fixed #30668 -- Made QuerySet.filter() raise NotSupportedError if any of source expressions is not filterable. 2019-08-02 15:06:00 +02:00
Mariusz Felisiak 806ba19bbf
Added Query.is_sliced property.
Previously, we used Query.can_filter() mainly to check if a query is
sliced what was confusing.
2019-07-25 20:45:55 +02:00
Mariusz Felisiak 7a42cfcfdc
Refs #30557 -- Fixed crash of ordering by ptr fields when Meta.ordering contains F() expressions.
Thanks Can Sarıgöl for the report.

Follow up to 8c5f9906c5.
2019-07-11 13:40:36 +02:00
Hasan Ramezani 8c5f9906c5 Fixed #30557 -- Fixed crash of ordering by ptr fields when Meta.ordering contains expressions. 2019-07-11 11:24:59 +02:00
Johannes Hoppe bc91f27a86 Refs #29444 -- Added support for fetching a returned non-integer insert values on Oracle.
This is currently not actively used, since the ORM will ask the
SQL compiler to only return auto fields.
2019-07-08 08:53:08 +02:00
can f03b7bd114 Fixed #28408 -- Added error message when updating with annotated expressions on joined fields.
Co-Authored-By: Simon Charette <charette.s@gmail.com>
2019-06-28 07:55:32 +02:00
can 2cbd3967e0 Fixed #29834 -- Fixed column mismatch crash with QuerySet.values()/values_list() and order_by() on combined querysets. 2019-06-19 12:11:28 +02:00
Mariusz Felisiak 14d026cccb
Fixed #30572 -- Prevented values()/values_list() on combined queryset from mutating the list of columns in querysets. 2019-06-19 10:44:53 +02:00
Mariusz Felisiak f6075fb333 Fixed #26192 -- Fixed crash of ordering by constants on PostgreSQL.
Thanks Simon Charette for the review.
2019-05-31 07:38:48 +02:00
Mariusz Felisiak f8b8b00f01
Fixed #30501 -- Preventing QuerySet.reverse() from mutating expressions in QuerySet.order_by and Meta.ordering. 2019-05-23 20:33:37 +02:00
ruchit2801 04042b2b44 Fixed #30463 -- Fixed crash of deprecation message when Meta.ordering contains expressions.
Regression in 1b1f64ee5a.
2019-05-18 19:29:00 +02:00
Rob 6b736dd074 Fixed #30349 -- Fixed QuerySet.exclude() on FilteredRelation.
Using annotated FilteredRelations raised a FieldError when coupled with
exclude(). This is due to not passing filtered relation fields to the
subquery created in split_exclude(). We fixed this issue by passing the
filtered relation data to the newly created subquery.

Secondly, in the case where an INNER JOIN is used in the excluded
subquery, the ORM would trim the filtered relation INNER JOIN in attempt
to simplify the query. This will also remove the ON clause filters
generated by the FilteredRelation. We added logic to not trim the INNER
JOIN if it is from FilteredRelation.
2019-05-07 14:47:50 +02:00
can 567b9928a3 Fixed #29692 -- Fixed removing ordering parts for multiline RawSQL expressions. 2019-05-03 07:37:37 +02:00
can 719b746620 Fixed #30412 -- Fixed crash when adding check constraints with OR'ed condition on Oracle and SQLite. 2019-04-30 12:32:27 +02:00
can c00b4dbda2 Fixed #29810 -- Fixed crash of select_related() on FilteredRelation with empty result. 2019-04-23 13:48:33 +02:00
can d87bd29c4f Fixed #30335, #29139 -- Fixed crash when ordering or aggregating over a nested JSONField key transform. 2019-04-18 07:16:50 +02:00
Simon Charette 0cb4062482 Refs #23758 -- Used RecursionError instead of RuntimeError to raise nested subquery errors.
RecursionError was introduced in Python 3.5 and subclasses RuntimeError.
2019-04-15 15:20:41 +02:00
Simon Charette c0969ee227 Refs #27149 -- Based recursive nested subquery detection on sys.getrecursionlimit().
This makes sure the test_avoid_infinite_loop_on_too_many_subqueries test
doesn't fail on systems with a non-default recursion limit.
2019-04-15 15:20:41 +02:00
Simon Charette 1ca825e4dc Fixed #30246 -- Reused annotation aliases references in aggregation filters.
Thanks Jan Baryła for the detailed report and the reduced test case.
2019-03-23 10:11:45 -04:00
Simon Charette 3f32154f40 Refs #30188 -- Avoided GROUP BY when aggregating over non-aggregates. 2019-03-23 10:11:41 -04:00
Simon Charette d1e9c25162 Refs #30188 -- Prevented double annotation of subquery when aggregated over.
Thanks Can Sarıgöl for the suggested trimming approach.
2019-03-23 09:50:42 -04:00
Simon Charette bdc07f176e Fixed #30188 -- Fixed a crash when aggregating over a subquery annotation. 2019-03-23 09:48:31 -04:00
Simon Charette f19a4945e1 Fixed #21703 -- Fixed a crash when excluding a related field with a F(). 2019-03-23 09:48:28 -04:00
Simon Charette 5935a9aead Removed obsolete Lookup hook to prepare rhs expressions.
After 3a505c70e7, all _prepare() methods
return self.
2019-03-21 20:54:21 -04:00
Simon Charette 3a505c70e7 Refs #27149, #29542 -- Simplified subquery parentheses wrapping logic. 2019-03-21 20:42:26 -04:00
Simon Charette 3543129822 Refs #27149 -- Moved subquery expression resolving to Query.
This makes Subquery a thin wrapper over Query and makes sure it respects
the Expression source expression API by accepting the same number of
expressions as it returns. Refs #30188.

It also makes OuterRef usable in Query without Subquery wrapping. This
should allow Query's internals to more easily perform subquery push downs
during split_exclude(). Refs #21703.
2019-03-21 20:36:31 -04:00
Simon Charette 9dc367dc10 Refs #30158 -- Added alias argument to Expression.get_group_by_cols(). 2019-03-21 18:47:46 -04:00
Hasan Ramezani 741ce81a42 Fixed #29619 -- Added field names to some FieldErrors. 2019-02-14 18:58:08 -05:00
Dan Davis 2bd8df243a Fixed #30184 -- Removed ellipsis characters from shell output strings.
Partially reverted 50b8493581 (refs #29654)
to avoid a crash when the user shell doesn't support non-ASCII characters.
2019-02-13 13:59:44 -05:00
Nick Pope 24b82cd201 Fixed #30159 -- Removed unneeded use of OrderedDict.
Dicts preserve order since Python 3.6.
2019-02-06 13:48:39 -05:00
Johannes Hoppe b131f9c79f Refs #29444 -- Renamed DatabaseFeatures.can_return_id* to be generic for other columns. 2019-01-30 15:31:56 -05:00
Mariusz Felisiak 5a5c77d55d
Fixed E117 and F405 flake8 warnings. 2019-01-30 13:06:09 +01:00
Tim Graham da5eb3d56c Refs #28370 -- Removed support for the context arg of Field.from_db_value() and Expression.convert_value().
Per deprecation timeline.
2019-01-17 10:52:19 -05:00
Simon Charette 9e5e5a657b Fixed #30044 -- Raised a FieldError on inherited field update attempts. 2019-01-16 14:09:51 -05:00
Sergey Fedoseev b86bb47818 Fixed #30093 -- Fixed ordering of combined queryset ordered by F expressions. 2019-01-14 10:28:11 -05:00
Mariusz Felisiak f9a33e3c3f Fixed #29932 -- Fixed combining compound queries with sub-compound queries on SQLite and Oracle. 2018-12-06 15:31:33 -05:00
Mariusz Felisiak d5f4ce9849
Fixed #29949 -- Refactored db introspection identifier converters.
Removed DatabaseIntrospection.table_name_converter()/column_name_converter()
and use instead DatabaseIntrospection.identifier_converter().

Removed DatabaseFeatures.uppercases_column_names.

Thanks Tim Graham for the initial patch and review and Simon Charette
for the review.
2018-11-21 09:06:50 +01:00
Jon Dufresne 2ba588e773 Refs #27795 -- Removed force_text() usage in db/models/sql/query.py. 2018-10-08 14:47:01 -04:00
Abhinav Patil bf01994a5c Fixed #29804 -- Added 'did you mean' suggestions for unsupported lookup error. 2018-10-01 19:03:10 -04:00
Sergey Fedoseev 8ef8bc0f64 Refs #28909 -- Simplifed code using unpacking generalizations. 2018-09-28 09:57:12 -04:00
Ramiro Morales 1b1f64ee5a Refs #14357 -- Deprecated Meta.ordering affecting GROUP BY queries.
Thanks Ramiro Morales for contributing to the patch.
2018-09-13 12:29:48 -04:00
Alexander Holmbäck f315d0423a Fixed #29727 -- Made nonexistent joins in F() raise FieldError.
Regression in 2162f0983d.
2018-09-08 09:40:33 -04:00
Mariusz Felisiak 39461a83c3
Fixed #29694 -- Fixed column mismatch crash with QuerySet.values() or values_list() after combining querysets with extra() with union(), difference(), or intersection().
Regression in 0b66c3b442.
2018-08-29 10:00:15 +02:00
Claude Paroz 50b8493581 Refs #29654 -- Replaced three dots with ellipsis character in output strings. 2018-08-22 09:13:58 -04:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) 7eb556a6c2 Removed unused enumerate in django/db/models/sql/datastructures.py. 2018-08-13 18:41:24 -04:00