Now intermediate states in the database_backwards are cached, similar to
the executor's migrate() (or _migrate_all_backwards()). This makes the
migration run much faster (O(n) instead of O(n^2) over number of
database_operations).
When Django reraises an exception, it sets the __cause__ attribute even
in Python 2, mimicking Python's 3 behavior for "raise Foo from Bar".
However, Python 3 also ensures that all exceptions have a __traceback__
attribute and thus the "traceback2" Python 2 module (backport of Python
3's "traceback" module) relies on the fact that whenever you have a
__cause__ attribute, the recorded exception also has a __traceback__
attribute.
This is breaking testtools which is using traceback2 (see
https://github.com/testing-cabal/testtools/issues/162).
This commit fixes this inconsistency by ensuring that Django sets
the __traceback__ attribute on any exception stored in a __cause__
attribute of a reraised exception.
The change partly goes back to the old behavior for forwards migrations
which should reduce the amount of memory consumption (#24745). However,
by the way the current state computation is done (there is no
`state_backwards` on a migration class) this change cannot be applied to
backwards migrations. Hence rolling back migrations still requires the
precomputation and storage of the intermediate migration states.
This improvement also implies that Django does not handle mixed
migration plans anymore. Mixed plans consist of a list of migrations
where some are being applied and others are being unapplied.
Thanks Andrew Godwin, Josh Smeaton and Tim Graham for the review as well
as everybody involved on the ticket that kept me looking into the issue.
The new attribute is checked when the `migrate --fake-initial` option
is used. initial will be set to True for all initial migrations (this
is particularly useful when initial migrations are split) as well as
for squashed migrations.
If the only manager on the model is the default manager defined
by Django (`objects = models.Manager()`), this manager will not
be added to the model state. If it is custom, it needs to be
passed to the model state.
The idea behind this change is, that AlterUniqueTogether,
AlterIndexTogether and AlterOrderWithRespectTo can always be moved after
an Add/Alter/Rename/RemoveField operation if they don't refer to the
respective field and are not empty sets / None.
Combined with the optimizations of duplicate AlterUniqueTogether,
AlterIndexTogether, and AlterOrderWithRespectTo operations from
128caa1e16, these operations are optimized
in a later round of the optimizer.
Thanks Tim Graham for the review.
This is a regression caused by introducing rendered migration states in
1aa3e09c20 and the _meta refactoring in fb48eb0581.
Thanks to Danilo Bargen for reporting the issue and Marten Kenbeek and
Tim Graham for triaging the bug and providing the initial test case.
This also prevents state modifications from corrupting previous states.
Previously, when a model defining a relation was unregistered first,
clearing the cache would cause its related models' _meta to be cleared
and would result in the old models losing track of their relations.
Calling Migration.mutate_state() now also allows to do in_place
mutations in case an intermediate state is thrown away later.
Thanks Anssi Kääriäinen for the idea, Ryan Hall for parts of the patch,
and Claude Paroz and Tim Graham for the review
Made MigrationGraph forwards_plan() and backwards_plan() fall back to an
iterative approach in case the recursive approach exceeds the recursion
depth limit.
This adds a new method, Apps.lazy_model_operation(), and a helper function,
lazy_related_operation(), which together supersede add_lazy_relation() and
make lazy model operations the responsibility of the App registry. This
system no longer uses the class_prepared signal.
Field.rel is now deprecated. Rel objects have now also remote_field
attribute. This means that self == self.remote_field.remote_field.
In addition, made the Rel objects a bit more like Field objects. Still,
marked ManyToManyFields as null=True.
Set apps.ready to False when rendering multiple models. This prevents
that the cache on Model._meta is expired on all models after each time a
single model is rendered. Prevented that Apps.clear_cache() refills the
cache on Apps.get_models(), so that the wrong value cannot be cached
when cloning a StateApps.
There's no reason to assume that sys.path[0] is an appropriate location
for generating code. Specifically that doesn't work with extend_sys_path
which puts the additional directories at the end of sys.path.
In order to create a new migrations module, instead of using an
arbitrary entry from sys.path, import as much as possible from the path
to the module, then create missing submodules from there.
Without this change, the tests introduced in the following commit fail,
which seems sufficient to prevent regressions for such a refactoring.
Switched from an adjancency list and uncached, iterative depth-first
search to a Node-based design with direct parent/child links and a
cached, recursive depth-first search. With this change, calculating
a migration plan for a large graph takes several seconds instead of
several hours.
Marked test `migrations.test_graph.GraphTests.test_dfs` as an expected
failure due to reaching the maximum recursion depth.
The new signature enables better support for routing RunPython and
RunSQL operations, especially w.r.t. reusable and third-party apps.
This commit also takes advantage of the deprecation cycle for the old
signature to remove the backward incompatibility introduced in #22583;
RunPython and RunSQL won't call allow_migrate() when when the router
has the old signature.
Thanks Aymeric Augustin and Tim Graham for helping shape up the patch.
Refs 22583.
Swapped out models don't have a _default_manager unless they have
explicitly defined managers. ModelState.from_model() now accounts for
this case and uses an empty list for managers if no explicit managers
are defined and a model is swapped out.
Instead of naively reloading only directly related models (FK, O2O, M2M
relationship) the project state needs to reload their relations as well
as the model changes as well. Furthermore inheriting models (and super
models) need to be reloaded in order to keep inherited fields in sync.
To prevent endless recursive calls an iterative approach is taken.
Previously Django only checked for the table name in CreateModel
operations in initial migrations and faked the migration automatically.
This led to various errors and unexpected behavior. The newly introduced
--fake-initial flag to the migrate command must be passed to get the
same behavior again. With this change Django will bail out in with a
"duplicate relation / table" error instead.
Thanks Carl Meyer and Tim Graham for the documentation update, report
and review.
The migrations autodetector now issues AlterModelOptions operations for
Meta.managed changes instead of DeleteModel + CreateModel.
Thanks iambibhas for the report and Simon and Markus for review.
Previous algo only worked if the first item was a part of the loop,
and you would get an infinite loop otherwise (see test).
To fix this, it was much easier to do a pre-pass.
A bonus is that you now get an error message that actually helps you debug
the dependency problem.
This reverts commit f36151ed16.
Adding kwargs to deconstructed objects does not achieve useful
forward-compatibility in general, since additional arguments are silently
dropped rather than having their intended effect. In fact, it can make the
failure more difficult to diagnose. Thanks Shai Berger for discussion.
This removes the concept of equality between operations to guarantee
compatilibity with Python 3.
Python 3 requires equality to result in identical object hashes. It's
impossible to implement a unique hash that preserves equality as
operations such as field creation depend on being able to accept
arbitrary dicts that cannot be hashed reliably.
Thanks Klaas van Schelven for the original patch in
13d613f800.
Amos Onn
Tim Graham
Simon Charette
Raphaël Hertzog
Ana Vojnovic
Andrei Fokau
Benjamin Wohlwend
Markus Holtermann
Dražen Odobašić
Alex Hill
Tyson Clugg
Jeremy Satterfield
Caio Ariede
Piper Merriam
Andrei Kulakov
Kai Richard Koenig
Andriy Sokolovskiy
Steadman
Carl Meyer
Yoong Kang Lim
Matt Westcott
Marten Kenbeek
Piotr Jakimiak
Claude Paroz
Luis Del Giudice
Matthew Wilkes
Christopher Luc
Patryk Zawadzki
Baptiste Mispelon
Anssi Kääriäinen
Aymeric Augustin
Loic Bistuer
Adam Taylor
Ulrich Petri
Daniel Pyrathon
Russell Keith-Magee
Oscar Ramirez
Josh Schneier
Andriy Sokolovskiy
Jon Dufresne
Gavin Wahl
Luke Plant
Andrzej Pragacz
Stratos Moros
twidi
Andrew Godwin
Niels Kouwenhoven
Klaas van Schelven
Berker Peksag
Tianyi Wang
Rudy Mutter
Thomas Chaumeny