Commit Graph

1770 Commits

Author SHA1 Message Date
Mariusz Felisiak 3a941230c8
Fixed #31584 -- Fixed crash when chaining values()/values_list() after Exists() annotation and aggregation on Oracle.
Oracle requires the EXISTS expression to be wrapped in a CASE WHEN in
the GROUP BY clause.

Regression in efa1908f66.
2020-05-14 15:07:08 +02:00
Mariusz Felisiak e536fa5ce1
Fixed #31579 -- Dropped support for PostgreSQL 9.5 and PostGIS 2.2. 2020-05-14 06:33:00 +02:00
Jon Dufresne d6aff369ad Refs #30116 -- Simplified regex match group access with Match.__getitem__().
The method has been available since Python 3.6. The shorter syntax is
also marginally faster.
2020-05-11 12:01:28 +02:00
sage 6789ded0a6 Fixed #12990, Refs #27694 -- Added JSONField model field.
Thanks to Adam Johnson, Carlton Gibson, Mariusz Felisiak, and Raphael
Michel for mentoring this Google Summer of Code 2019 project and
everyone else who helped with the patch.

Special thanks to Mads Jensen, Nick Pope, and Simon Charette for
extensive reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2020-05-08 07:23:31 +02:00
Mariusz Felisiak f97f71f592 Refs #12990 -- Bumped mysqlclient requirement to >= 1.4.0.
MySQLdb.constants.FIELD_TYPE.JSON was added in mysqlclient 1.4.0rc2.
2020-05-08 06:14:42 +02:00
Adam Johnson 1c2c6f1b51
Fixed a/an typos in "SQL" usage. 2020-05-06 06:35:26 +02:00
Ian Foote c226c6cb32 Fixed #20581 -- Added support for deferrable unique constraints. 2020-04-30 10:43:50 +02:00
Hannes Ljungberg 104493311e Refs #30913 -- Added BaseDatabaseSchemaEditor._index_condition_sql(). 2020-04-24 07:10:26 +02:00
David Wobrock 533b208775 Fixed #29224 -- Fixed removing index_together indexes if exists unique_together constraint on the same fields. 2020-04-23 12:28:51 +02:00
Jon Dufresne 75866b93cc Fixed #31479 -- Added support to reset sequences on SQLite. 2020-04-21 10:41:47 +02:00
Simon Charette 5220ca8d8a Refs #30591 -- Adjusted table rebuild for non-pk relationship on SQLite.
The existing code was only accounting for primary key changes and not
all unique key fields that can be referenced.
2020-04-21 08:37:47 +02:00
Simon Charette a548280857 Fixed #31064 -- Recreated auto-created many-to-many tables on primary key data type change on SQLite.
Both local and remote auto-created many-to-many relationships were
affected.
2020-04-21 08:37:40 +02:00
Simon Charette 330c0ed7c4 Stopped rebuilding referenced tables multiple times on SQLite alterations. 2020-04-20 13:13:57 +02:00
Jon Dufresne 505fec6bad Capitalized Unicode in docs, strings, and comments. 2020-04-20 12:10:33 +02:00
Jon Dufresne 5673d4b102
Fixed #31477 -- Removed "using" argument from DatabaseOperations.execute_sql_flush(). 2020-04-20 07:49:35 +02:00
Jon Dufresne 75410228df Fixed #31473 -- Made sql_flush() use RESTART IDENTITY to reset sequences on PostgreSQL.
The sql_flush() positional argument sequences is replaced by the boolean
keyword-only argument reset_sequences. This ensures that the old
function signature can't be used by mistake when upgrading Django. When
the new argument is True, the sequences of the truncated tables will
reset. Using a single boolean value, rather than a list, allows making a
binary yes/no choice as to whether to reset all sequences rather than a
working on a completely different set.
2020-04-17 11:57:24 +02:00
Jon Dufresne 8005829bb9 Simplified DatabaseOperations.sql_flush() on Oracle and PostgreSQL.
Added early return to decrease an indentation level.
2020-04-17 10:46:19 +02:00
Mariusz Felisiak eeab63e57e
Refs #26552 -- Made reloading the database for tests check only loaded tables constraints. 2020-04-17 08:14:27 +02:00
Adam Johnson 5b884d45ac Fixed #29501 -- Allowed dbshell to pass options to underlying tool. 2020-04-14 14:02:51 +01:00
Mariusz Felisiak 9cab261427
Refs #31411 -- Used RENAME COLUMN on MySQL 8.0.4+.
MySQL 8.0.3 added support for this syntax but also imposed a
restriction against ALTER TABLE RENAME on tables in a foreign key
relationship if a LOCK TABLES was active which has been lifted in MySQL
8.0.4+.
2020-04-09 11:16:28 +02:00
Sergey Fedoseev 026719cf17 Fixed #31030 -- Registered SQLite functions as deterministic on Python 3.8+. 2020-04-06 11:26:00 +02:00
c-bata 89032876f4 Fixed #31275 -- Optimized sql_flush() without resetting sequences on MySQL.
Co-Authored-By: Simon Charette <charettes@users.noreply.github.com>
2020-04-02 12:57:53 +02:00
Matthijs Kooijman 75520e1767 Refs #31051 -- Optimized serialize_db_to_string() by avoiding creation of models list. 2020-04-02 11:38:23 +02:00
Matthijs Kooijman 289d0ec6fd Refs #31051 -- Fixed reloading the database with circular related objects and natural keys for tests.
Made deserialize_db_from_string() do not sort dependencies.

deserialize_db_from_string() doesn't use natural keys, so there is no
need to sort dependencies in serialize_db_to_string(). Moreover,
sorting models cause issues for circular dependencies.
2020-04-02 11:35:33 +02:00
Hasan Ramezani 1fe3e24264 Fixed #31411 -- Used RENAME COLUMN on MariaDB 10.5.2+. 2020-04-02 06:56:15 +02:00
Adam Johnson 93ed71e058 Fixed #31403 -- Added support for returning fields from INSERT statements on MariaDB 10.5+. 2020-03-31 07:25:50 +02:00
Adam Johnson c06d7c9239 Refs #31403 -- Made SQLInsertCompiler.execute_sql() always return an iterable of rows.
List of tuples.
2020-03-31 07:12:11 +02:00
Tim Graham 8bebb8f126 Fixed #31399 -- Made SchemaEditor._alter_field() ignore foreign key constraints if unsupported. 2020-03-25 20:13:29 +01:00
Hannes Ljungberg f3da09df0f Fixed #31396 -- Added binary XOR operator to F expressions. 2020-03-25 10:16:30 +01:00
Shipeng Feng 8fe2447a01
Fixed #31392 -- Avoided unnecessary SchemaEditor.effective_default() calls when altering a field. 2020-03-24 13:09:43 +01:00
Rohit 2695ac8e04 Fixed #31144 -- Relaxed system check for max_length of CharFields on MySQL/MariaDB by turning into a warning. 2020-03-20 12:28:10 +01:00
Mariusz Felisiak 6e026aec5f
Refs #29548 -- Mentioned MariaDB in database system checks. 2020-03-19 09:42:04 +01:00
Simon Charette 9f07f27124 Fixed #31376 -- Optimized nulls ordering when possible on SQLite and MySQL.
Both backends order NULLs first on ascending ordering and last on
descending ordering which makes ORDER BY IS (NOT)? NULL wasteful when
asc(nulls_first) and desc(nulls_last) are used since it prevents indice
usage.
2020-03-18 06:43:49 +01:00
Claude Paroz e663f695fb Fixed #31359 -- Deprecated get_random_string() calls without an explicit length. 2020-03-11 13:16:44 +01:00
shankarj67 df4d622cac Fixed #31345 -- Added BaseDatabaseIntrospection.get_relations(). 2020-03-06 08:53:32 +01:00
Jon Dufresne 769cee5252 Fixed #31327 -- Deprecated providing_args argument for Signal. 2020-03-05 09:38:52 +01:00
aryan 427a7e419b Fixed #31333 -- Added BaseDatabaseIntrospection.get_table_description(). 2020-03-04 12:26:31 +01:00
Mariusz Felisiak a19505eb2e Refs #31331 -- Added DatabaseWrapper.sql_mode to MySQL. 2020-03-03 11:25:37 +01:00
Hans Aarne Liblik a21f7b91db Fixed #31301 -- Fixed crash of QuerySet.bulk_create() with mixed empty and set ForeignKeys to AutoFields on Oracle. 2020-02-28 12:56:03 +01:00
Simon Charette 41ebe60728 Fixed #31312 -- Properly ordered temporal subtraction params on MySQL.
Regression in 9bcbcd599a.

Thanks rick2ricks for the report.
2020-02-27 08:50:55 +01:00
Mariusz Felisiak 2a038521c4
Fixed #31271 -- Preserved ordering when unifying query parameters on Oracle.
This caused misplacing parameters in logged SQL queries.

Regression in 79065b55a7.

Thanks Hans Aarne Liblik for the report.
2020-02-18 11:45:12 +01:00
Taoup cbb6531e5b Fixed #31228 -- Reallowed aggregates to be used with multiple expressions and no DISTINCT on SQLite.
Regression in bc05547cd8.

Thanks Andy Terra for the report.
2020-02-18 10:32:23 +01:00
Matthijs Kooijman 98f23a8af0 Fixed #26552 -- Deferred constraint checks when reloading the database with data for tests.
deserialize_db_from_string() loads the full serialized database
contents, which might contain forward references and cycles. That
caused IntegrityError because constraints were checked immediately.

Now, it loads data in a transaction with constraint checks deferred
until the end of the transaction.
2020-02-14 13:18:06 +01:00
Jon Dufresne 3259983f56 Fixed #31233 -- Closed database connections and cursors after use. 2020-02-06 15:35:23 +01:00
Jon Dufresne f48f671223 Refs #31233 -- Changed DatabaseWrapper._nodb_connection to _nodb_cursor().
It is now a method instead of a property and returns a context manager
that yields a cursor on entry and closes the cursor and connection upon
exit.
2020-02-06 15:29:38 +01:00
Nick Pope 335c9c94ac Simplified imports from django.db and django.contrib.gis.db. 2020-02-04 13:20:06 +01:00
Jon Dufresne 958977f662 Fixed #31212 -- Updated psycopg links to HTTPS and new location. 2020-01-29 06:25:40 +01:00
Tim Graham 227d0c7365 Fixed #31183 -- Added a feature flag for "<db> only supports UNBOUNDED together with PRECEDING and FOLLOWING". 2020-01-20 20:03:35 +01:00
Jon Dufresne 26be703fe6 Changed re-raising an exception to use bare raise syntax where appropriate. 2020-01-20 11:44:28 +01:00
Simon Charette 9bcbcd599a Fixed #31133 -- Fixed crash when subtracting against a subquery annotation.
The subtract_temporals() database operation was not handling expressions
returning SQL params in mixed database types.

Regression in 3543129822.

Thanks Reupen Shah for the report.
2020-01-03 10:35:08 +01:00
Mariusz Felisiak 22ce5d0031
Fixed #31106 -- Fixed migrations crash on PostgreSQL 10+ when adding FK constraints inline and changing data.
This allows adding foreign key constraints inline and changing data in
the same migration on PostgreSQL 10+.

Regression in 738faf9da2.

Thanks Janne Rönkkö for the report and Simon Charette for the
implementation idea and review.
2019-12-23 23:28:59 +01:00
James Turk ff00a05347 Fixed #31088 -- Added support for websearch searching in SearchQuery. 2019-12-16 14:59:59 +01:00
Zeynel Özdemir 9d40b6bbf4 Fixed #31076 -- Fixed dbshell crash on Windows with Python < 3.8.
subprocess.run()'s args parameter accepts path-like objects on Windows
since Python 3.8.
2019-12-16 11:20:41 +01:00
Sergey Fedoseev 88637064b3 Refs #28373 -- Stopped setting tzinfo in typecast_timestamp().
Unnecessary since cef3f2d3c6. When
USE_TZ=True, conn_tznames is always passed to methods that use
typecast_timestamp(). Therefore, it can return naive datetimes.
2019-12-11 13:59:49 +01:00
Aymeric Augustin c06492dd87 Fixed #23524 -- Allowed DATABASES['TIME_ZONE'] option on PostgreSQL. 2019-12-04 18:22:08 +01:00
Aymeric Augustin ad88524e4d Refs #23524 -- Unified BaseDatabaseWrapper.timezone.
There was a special case in this property to return None when the
database backend supports time zone. However, only the PostgreSQL
backend supports time zones and it never uses this property.
2019-12-04 18:07:51 +01:00
Farhaan Bukhsh 664c98f1f8 Fixed #30413 -- Fixed test database signature on SQLite when test database name is provided.
Previously, the same signature was created for multiple in-memory
databases on SQLite when they had tests databases names
DATABASES['TEST']['NAME'].
2019-11-21 19:50:24 +01:00
Simon Charette 37e6c5b79b Refs #25367 -- Moved conditional expression wrapping to the Exact lookup. 2019-11-21 11:56:35 +01:00
Jon Dufresne f5ebdfce5c Fixed #25388 -- Added an option to allow disabling of migrations during test database creation. 2019-11-20 20:42:38 +01:00
Caio Ariede 555bebe774 Fixed #30987 -- Added models.PositiveBigIntegerField. 2019-11-19 09:34:11 +01:00
Jon Dufresne 57a3d96ff5 Replaced unnecessary str()/bytes() calls with literals. 2019-11-18 15:30:10 +01:00
Mariusz Felisiak 8685e764ef Fixed #30986 -- Fixed queryset crash when filtering against boolean RawSQL expressions on Oracle. 2019-11-18 08:50:09 +01:00
Hannes Ljungberg 6d590bcf1f Fixed #30961 -- Fixed spaces in columns list SQL generated for indexes. 2019-11-08 08:52:47 +01:00
Nick Pope 02983c5242 Fixed #30943 -- Added BloomIndex to django.contrib.postgres. 2019-11-07 11:23:53 +01:00
Jon Dufresne 77aa74cb70 Refs #29983 -- Added support for using pathlib.Path in all settings. 2019-11-07 10:26:22 +01:00
Mariusz Felisiak a20ea33ca6
Fixed DatabaseFeatures.has_select_for_update_nowait on MariaDB 10.3+.
Thanks Kola Erinoso for the report.
2019-11-05 10:18:47 +01:00
Nick Pope 7286eaf681 Refs #13312 -- Simplified handling of nulls ordering on MySQL.
MySQL & MariaDB support the standard IS NULL and IS NOT NULL so
the same workaround used for NULLS FIRST and NULLS LAST that is
used for SQLite < 3.30.0 can be used.

Thanks Simon Charette for the discussion.
2019-11-04 14:47:58 +01:00
Simon Charette 377c7cb2f7 Refs #23576 -- Disabled MySQL multi-alias deletion path on MariaDB 10.3.2+. 2019-10-31 08:14:30 +01:00
Hasan Ramezani e3d0b4d550 Fixed #30899 -- Lazily compiled import time regular expressions. 2019-10-29 09:22:26 +01:00
Nick Pope 55df1750be Refs #30897 -- Added support for ANALYZE option to Queryset.explain() on MariaDB and MySQL 8.0.18+. 2019-10-24 15:13:26 +02:00
Nick Pope 742961332e Refs #30897 -- Added support for TREE format to Queryset.explain() on MySQL 8.0.16+. 2019-10-24 15:13:23 +02:00
Simon Charette 7acef095d7 Fixed #23576 -- Implemented multi-alias fast-path deletion in MySQL backend.
This required moving the entirety of DELETE SQL generation to the
compiler where it should have been in the first place and implementing
a specialized compiler on MySQL/MariaDB.

The MySQL compiler relies on the "DELETE table FROM table JOIN" syntax
for queries spanning over multiple tables.
2019-10-24 12:24:53 +02:00
Mariusz Felisiak e645f27907
Fixed DatabaseFeatures.update_can_self_select on MariaDB 10.3.2+. 2019-10-24 12:18:13 +02:00
Hannes Ljungberg fa5f3291e7 Fixed #30903 -- Fixed migrations crash on PostgreSQL when adding Index with opclasses and ordering. 2019-10-24 09:33:14 +02:00
Nick Pope 6e9189c080 Refs #30897 -- Added test for SETTINGS option to Queryset.explain() on PostgreSQL 12+. 2019-10-23 14:07:18 +02:00
Hasan Ramezani ef4beafa2c Refs #28816 -- Prevented silencing data loss when decreasing CharField.max_length for ArrayField.base_field on PostgreSQL. 2019-10-17 12:50:53 +02:00
Mariusz Felisiak 187a64608d Fixed #30885 -- Dropped support for MariaDB 10.1. 2019-10-16 18:40:48 +02:00
Mariusz Felisiak 4a849b6701
Fixed #30859 -- Fixed DatabaseFeatures.supports_aggregate_filter_clause on SQLite 3.30.1+. 2019-10-15 20:55:49 +02:00
Mariusz Felisiak 02c63b8f2f
Refs #26608 -- Fixed DatabaseFeatures.supports_frame_range_fixed_distance on SQLite 3.28+, MariaDB 10.2+, and MySQL 8.0.2+. 2019-10-09 13:07:50 +02:00
Anatol Ulrich 8ed6788aa4 Fixed #30821 -- Added ExtractIsoWeekYear database function and iso_week_day lookup. 2019-10-02 10:41:34 +02:00
Mariusz Felisiak 67e7dffe95 Fixed typo in BulkInsertMapper constant name. 2019-10-01 14:46:10 +02:00
Ahmet Kucuk dc890bef5a Fixed #30510 -- Fixed crash of QuerySet.bulk_create() with mixed-length texts on Oracle.
Text with more than 4000 characters must be set to as a CLOB on Oracle
what caused a mixed datatype error (ORA-01790) when shorter text
appeared in the same operation.
2019-10-01 12:38:58 +02:00
Mariusz Felisiak 580e644f24
Fixed #30800 -- Fixed migrations crash when altering a field with custom db_type().
Regression in 1378d665a1.
2019-09-25 12:32:24 +02:00
Mariusz Felisiak 01104368ff Refs #29444 -- Removed redundant DatabaseFeatures.can_return_multiple_columns_from_insert.
Unnecessary since b31e63879e.
2019-09-24 10:37:22 +02:00
Johannes Hoppe b31e63879e Fixed #29444 -- Allowed returning multiple fields from INSERT statements on Oracle. 2019-09-24 10:22:43 +02:00
Andrew 3346b78a8a Fixed #30786 -- Used CONVERT_TZ to check if the time zone definitions are installed on MySQL.
Replaced a timezone check in the MySQL backend with one that doesn't
require access to the mysql.time_zone database.
2019-09-20 08:17:27 +02:00
Adnan Umer 241deed259 Fixed #30591 -- Fixed recreation of foreign key constraints on MySQL when altering type of referenced unique field.
Thanks Mariusz Felisiak for tests and Matthijs Kooijman for
investigation and initial patch.
2019-09-11 11:21:08 +02:00
Vojtech Bocek b1d37fea8f Fixed #28107 -- Added DatabaseFeatures.allows_group_by_selected_pks_on_model() to allow enabling optimization for unmanaged models. 2019-09-09 14:04:46 +02:00
Johannes Hoppe 7254f1138d Refs #29444 -- Allowed returning multiple fields from INSERT statements on PostgreSQL.
Thanks Florian Apolloner, Tim Graham, Simon Charette, Nick Pope, and
Mariusz Felisiak for reviews.
2019-09-09 10:51:14 +02:00
Johannes Hoppe 736e7d44de Refs #29444 -- Fixed DateField constructor in db.backends.oracle.utils.InsertVar. 2019-09-09 10:51:07 +02:00
Mariusz Felisiak addabc492b
Fixed DatabaseFeatures.can_introspect_check_constraints on MariaDB < 10.2.22, 10.3.0 - 10.3.9.
Regression in e2c6a0858d.
2019-09-06 11:53:54 +02:00
Mariusz Felisiak e2c6a0858d Fixed #30750 -- Added support for check constraints on MySQL 8.0.16+. 2019-09-05 09:18:34 +02:00
Johannes Hoppe 003bb34b21 Refs #29444 -- Made db.backends.oracle.utils.InsertVar use str as default. 2019-09-03 10:34:55 +02:00
Matthew Schinckel 4137fc2efc Fixed #25367 -- Allowed boolean expressions in QuerySet.filter() and exclude().
This allows using expressions that have an output_field that is a
BooleanField to be used directly in a queryset filters, or in the
When() clauses of a Case() expression.

Thanks Josh Smeaton, Tim Graham, Simon Charette, Mariusz Felisiak, and
Adam Johnson for reviews.

Co-Authored-By: NyanKiyoshi <hello@vanille.bid>
2019-08-29 09:45:29 +02:00
Mariusz Felisiak 579909a13f
Refs #30591 -- Fixed introspection of check and unique column constraints on MariaDB.
Unnamed unique and check columns constraints have the same name as
a column. Ensure uniqueness by using custom names.

Thanks Adnan Umer for the report.
2019-08-26 09:15:37 +02:00
Claude Paroz 9386586f31 Replaced subprocess commands by run() wherever possible. 2019-08-23 10:53:36 +02:00
Nasir Hussain 6b16c91157 Fixed #30712 -- Allowed BLOB/TEXT defaults on MySQL 8.0.13+. 2019-08-22 12:23:10 +02:00
Mads Jensen 85ac838d9e Fixed #21039 -- Added AddIndexConcurrently/RemoveIndexConcurrently operations for PostgreSQL.
Thanks to Simon Charettes for review.

Co-Authored-By: Daniel Tao <daniel.tao@gmail.com>
2019-08-21 13:10:06 +02:00
Nick Pope 21e559495b Fixed #29979, Refs #17337 -- Extracted AutoField field logic into a mixin and refactored AutoFields.
This reduces duplication by allowing AutoField, BigAutoField and
SmallAutoField to inherit from IntegerField, BigIntegerField and
SmallIntegerField respectively. Doing so also allows for enabling the
max_length warning check and minimum/maximum value validation for auto
fields, as well as providing a mixin that can be used for other possible
future auto field types such as a theoretical UUIDAutoField.
2019-08-20 09:22:25 +02:00
Adam Johnson 7da6a28a44 Fixed #27676 -- Allowed BLOB/TEXT defaults on MariaDB 10.2.1+. 2019-08-16 11:39:37 +02:00