This commit touchs various parts of the code base and test framework. Any
found usage of opening a cursor for the sake of initializing a connection
has been replaced with 'ensure_connection()'.
This is the result of Christopher Medrela's 2013 Summer of Code project.
Thanks also to Preston Holmes, Tim Graham, Anssi Kääriäinen, Florian
Apolloner, and Alex Gaynor for review notes along the way.
Also: Fixes#8579, fixes#3055, fixes#19844.
In get_commands, setup() might already have been called, for example
when the management command is called through call_command. Moving
setup() to ManagementUtility so as it is only called when the command
is run from command line.
Now that the refactorings are complete, it isn't particularly useful any
more, nor very well named. Let's keep the API as simple as possible.
Fixed#21689.
Since it triggers imports, it shouldn't be done lightly.
This commit adds a public API for doing it explicitly, django.setup(),
and does it automatically when using manage.py and wsgi.py.
Returning None on errors required unpythonic error checking and was
inconsistent with get_app_config.
get_model was a private API until the previous commit, but given that it
was certainly used in third party software, the change is explained in
the release notes.
Applied the same change to get_registered_model, which is a new private
API introduced during the recent refactoring.
This is to provide a consistent interface (namely bytes) for the smtp
backend which after all sends bytes over the wire; encoding with as_string
yields different results since mails as unicode are not really specified.
as_string stays for backwardscompatibilty mostly and some debug outputs.
But keep in mind that the output doesn't match as_bytes!
The last component of the dotted path to the application module is
consistently referenced as the application "label". For instance it's
AppConfig.label. appname could be confused with AppConfig.name, which is
the full dotted path.
- Tested consistency the current app_configs instead of INSTALLED_APPS.
- Considered applications added with _with_app as available.
- Added docstrings.
Took this opportunity to change get_app[s] to only consider applications
containing a model module as that seems slightly more backwards compatible.
Since callers that care about models pass only_with_models_module=True,
this has very few consequences. Only AppCommand needed a change.
Adjusted several tests that used it to add apps to the app cache and
then attempted to remove them by manipulating attributes directly.
Also renamed invalid_models to invalid_models_tests to avoid clashing
application labels between the outer and the inner invalid_models
applications.
It was called _populate() before I renamed it to populate(). Since it
has been superseded by populate_models() there's no reason to keep it.
Removed the can_postpone argument of load_app() as it was only used by
populate(). It's a private API and there's no replacement. Simplified
load_app() accordingly. Then new version behaves exactly like the old
one even though it's much shorter.
First stage imports app modules. It doesn't catch import errors. This
matches the previous behavior and keeps the code simple.
Second stage import models modules. It catches import errors and retries
them after walking through the entire list once. This matches the
previous behavior and seems useful.
populate_models() is intended to be equivalent to populate(). It isn't
wired yet. That is coming in the next commit.
Since applications that aren't installed no longer have an application
configuration, it is now always True in practice.
Provided an abstraction to temporarily add or remove applications as
several tests messed with app_config.installed to achieve this effect.
For now this API is _-prefixed because it looks dangerous.
Got rid of AppConfig._stub. As a side effect, app_cache.app_configs now
only contains entries for applications that are in INSTALLED_APPS, which
is a good thing and will allow dramatic simplifications (which I will
perform in the next commit). That required adjusting all methods that
iterate on app_configs without checking the "installed" flag, hence the
large changes in get_model[s].
Introduced AppCache.all_models to store models:
- while the app cache is being populated and a suitable app config
object to register models isn't available yet;
- for applications that aren't in INSTALLED_APPS since they don't have
an app config any longer.
Replaced get_model(seed_cache=False) by registered_model() which can be
kept simple and safe to call at any time, and removed the seed_cache
argument to get_model[s]. There's no replacement for that private API.
Allowed non-master app caches to go through populate() as it is now
safe to do so. They were introduced in 1.7 so backwards compatibility
isn't a concern as long as the migrations framework keeps working.
Improved Andrew's hack to create temporary app caches to handle
migrations. Now the main app cache has a "master" flag set to True
(which is a non-default keyword argument, thus unlikely to be used by
mistake). Other app cache instances have "master" set to False.
The only sanctioned way to access the app cache is by importing
django.core.apps.app_cache.
If you were instanciating an app cache and relying on the Borg pattern,
you'll have to refactor your code.
Several parts of Django call get_apps() with a comment along this lines
of "this has the side effect of calling _populate()". I fail to see how
this is better than just calling populate()!
Since the original ones in django.db.models.loading were kept only for
backwards compatibility, there's no need to recreate them. However, many
internals of Django still relied on them.
They were also imported in django.db.models. They never appear in the
documentation, except a quick mention of get_models and get_app in the
1.2 release notes to document an edge case in GIS. I don't think that
makes them a public API.
This commit doesn't change the overall amount of global state but
clarifies that it's tied to the app_cache object instead of hiding it
behind half a dozen functions.
The goal of this change is twofold; firstly, matching the behavior of Django 1.6
and secondly, an AttributeError is more informative than an obscure ValueError
about mismatching sequence lengths.
Refs #20867.
Previously when collecting static files, the directories would receive permissions
from the global umask. Now the default permission comes from FILE_UPLOAD_DIRECTORY_PERMISSIONS
and there's an option to specify the permissions by subclassing any of the
static files storage classes and setting the directory_permissions_mode parameter.
Removed multiple locales separated by commas variation (that wasn't
working as documented) in favor of simply allowing use of the
``--locale``/``-l`` options more than once for ``makemessages`` and
``compilemessages``.
Thanks Romain Beylerian for the report and Claude, Simon for their help.
8750296918 from stable/1.6.x.
Thanks Curtis Malony and Florian Apolloner.
Squashed commit of the following:
commit 3380495e93
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 14:18:07 2013 +0100
Looked up the template_fragments cache at runtime.
commit 905a74f52b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 14:19:48 2013 +0100
Removed all uses of create_cache.
Refactored the cache tests significantly.
Made it safe to override the CACHES setting.
commit 35e289fe92
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 12:23:57 2013 +0100
Removed create_cache function.
commit 8e274f747a
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 12:04:52 2013 +0100
Updated docs to describe a simplified cache backend API.
commit ee7eb0f73e
Author: Curtis Maloney <curtis@tinbrain.net>
Date: Sat Oct 19 09:49:24 2013 +1100
Fixed#21012 -- Thread-local caches, like databases.
Replaced the custom, untested memoize with a similar decorator from Python's
3.2 stdlib. Although some minor performance degradation (see ticket), it is
expected that in the long run lru_cache will outperform memoize once it is
implemented in C.
Thanks to EvilDMP for the report and Baptiste Mispelon for the idea of
replacing memoize with lru_cache.
Filtering out static file requests in runserver has been judged
arbitrary and can hide some debugging-related activity.
Thanks Roy Smith for the report and Aymeric Augustin for the
review.
* Safer for use in multiprocess environments
* Better random culling
* Cache files use less disk space
* Safer delete behavior
Also fixed#15806, fixed#15825.
Previously, when collecting static files, the files would receive permission
from FILE_UPLOAD_PERMISSIONS. Now, there's an option to give different
permission from uploaded files permission by subclassing any of the static
files storage classes and setting the file_permissions_mode parameter.
Thanks dblack at atlassian.com for the suggestion.
This shows itself with Python 3 under Windows where UTF-8 usually isn't
the default file I/O encoding and caused one failure and three errors
in our test suite under that platform setup.
When listing available management commands, only core commands are
listed if settings have any error. This commit adds a note in this
case so errors are not totally silently skipped.
Thanks Peter Davis for the report.
`HttpRequest.scheme` is `https` if `settings.SECURE_PROXY_SSL_HEADER` is
appropriately set and falls back to `HttpRequest._get_scheme()` (a hook
for subclasses to implement) otherwise.
`WSGIRequest._get_scheme()` makes use of the `wsgi.url_scheme` WSGI
environ variable to determine the request scheme.
`HttpRequest.is_secure()` simply checks if `HttpRequest.scheme` is
`https`.
This provides a way to check the current scheme in templates, for example.
It also allows us to deal with other schemes.
Thanks nslater for the suggestion.
Added ``--natural-foreign`` and ``--natural-primary`` options and
deprecated the ``--natural`` option to the ``dumpdata`` management
command.
Added ``use_natural_foreign_keys`` and ``use_natural_primary_keys``
arguments and deprecated the ``use_natural_keys`` argument to
``django.core.serializers.Serializer.serialize()``.
Thanks SmileyChris for the suggestion.
Literals from source files with Django template language syntax don't
have a '.py' suffix anymore.
Also, the '.\' prefix is preserved to respect GNU gettext behavior on
that platform.
Refs #16903.
The precision of time.time() is OS specific and it is possible for the
resolution to be low enough to allow reading a cache key previously set
with a timeout of 0.
DatabaseCache uses raw cursors to bypass the ORM. This prevents it from
being used by database backends that require special handling of datetime
values.
There is no easy way to test this, so no tests added.
Our WSGIServer rewrapped the socket errors from server_bind into
WSGIServerExceptions, which is used later on to provide nicer
error messages in runserver and used by the liveserver to see if
the port is already in use. But wrapping server_bind isn't enough since
it only binds to the socket, socket.listen (which is called from
server_activate) could also raise "Address already in use".
Instead of overriding server_activate too I chose to just catch socket
errors, which seems to make more sense anyways and should be more robust
against changes in wsgiref.
- Noted that this does not allow for reading and writing the same open
file in different processes under Windows.
- Noted that the keyword arguments to NamedTemporaryFile no longer
match the Python version.
Thanks mitsuhiko for the report.
Non-ASCII values are supported. Non-ASCII keys still aren't, because the
current parser mangles them. That's another bug.
Moved the get_serializer() call within the condition that checks public
serializers. This will allow exceptions other than
SerializerDoesNotExist to be raised in order to provide the caller with
useful information, e.g when pyyaml is not installed.
The serializer definitely exists, but the dependent yaml module may not
be installed. The register_serializer() function will catch exceptions
and will stub in a fake serializer object that will raise the exception
when the serializer is used.
- TemporaryFile now minimally mocks the API of the Python standard
library class tempfile.NamedTemporaryFile to avoid AttributeError
exceptions.
- The symbol django.core.files.NamedTemporaryFile is actually assigned
as a different class on different operating systems.
- The bug only occurred if Django is running on Windows, hence why it
was hard to diagnose.
IPython 1.0 introduces an actual stable public API function
for starting a normal (non-embedded) IPython session.
This is an official public API, which is promised to survive implementation changes.
Also moved its documentation to the 'advanced' section. It doesn't
belong to the 'overview'. Same for TransactionTestCase.reset_sequences.
When available_apps is set, after a TransactionTestCase, the database
is now totally empty. post_syncdb is fired at the beginning of the next
TransactionTestCase.
Refs #20483.
This allows using flush on a subset of the tables without having to
manually cascade to all tables with foreign keys to the tables being
truncated, when they're known to be empty.
On databases where truncate is implemented with DELETE FROM, this
doesn't make a difference. The cascade is allowed, not mandatory.
SuspiciousOperations have been differentiated into subclasses, and
are now logged to a 'django.security.*' logger. SuspiciousOperations
that reach django.core.handlers.base.BaseHandler will now return a 400
instead of a 500.
Thanks to tiwoc for the report, and Carl Meyer and Donald Stufft
for review.
A decorator is easier to apply to CBVs. Backwards compatibility isn't an
issue here, except for people running on a recent clone of master.
Fixed a few minor problems in the transactions docs while I was there.
If the fixture doesn't exist, loaddata will output a warning.
The fixture named "initial_data" is exceptional though; if it
doesn't exist, the warning is not emitted. This allows syncdb and
flush management commands to attempt to load it without causing
spurious warnings.
Thanks to Derega, ptone, dirigeant and d1ffuz0r for contributions
to the ticket.
The fixture named "initial_data" is exceptional though; if it
doesn't exist, the error is not raised. This allows syncdb and
flush management commands to attempt to load it without causing
an error if it doesn't exist.
Thanks Anssi for haggling until I implemented this.
This change alleviates the need for atomic_if_autocommit. When
autocommit is disabled for a database, atomic will simply create and
release savepoints, and not commit anything. This honors the contract of
not doing any transaction management.
This change also makes the hack to allow using atomic within the legacy
transaction management redundant.
None of the above will work with SQLite, because of a flaw in the design
of the sqlite3 library. This is a known limitation that cannot be lifted
without unacceptable side effects eg. triggering arbitrary commits.
Replaced them with per-database options, for proper multi-db support.
Also toned down the recommendation to tie transactions to HTTP requests.
Thanks Jeremy for sharing his experience.
Since "unless managed" now means "if database-level autocommit",
committing or rolling back doesn't have any effect.
Restored transactional integrity in a few places that relied on
automatically-started transactions with a transitory API.
enter_transaction_management() was nearly always followed by managed().
In three places it wasn't, but they will all be refactored eventually.
The "forced" keyword argument avoids introducing behavior changes until
then.
This is mostly backwards-compatible, except, of course, for managed
itself. There's a minor difference in _enter_transaction_management:
the top self.transaction_state now contains the new 'managed' state
rather than the previous one. Django doesn't access
self.transaction_state in _enter_transaction_management.
Fixes#19584.
This implies stop storing file path command line arguments in envvars as
a security measure to start relying on with Popen's shell=False instead,
and addition of an 'utils' module.
Thanks kmichel_wgs for the report.
Change patch from https://code.djangoproject.com/ticket/5568
to work on modern Django.
Add special case for MySQL which has different syntax for DROP INDEX.
Add unit tests for the new functionality.
Squashed commit of:
commit 508ec9144b35c50794708225b496bde1eb5e60aa
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 22:50:55 2013 +0100
Tweaked default settings file.
* Explained why BASE_DIR exists.
* Added a link to the database configuration options, and put it in its
own section.
* Moved sensitive settings that must be changed for production at the
top.
commit 6515fd2f1aa73a86dc8dbd2ccf512ddb6b140d57
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 14:35:21 2013 +0100
Documented the simplified app & project templates in the changelog.
commit 2c5b576c2ea91d84273a019b3d0b3b8b4da72f23
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 13:59:27 2013 +0100
Minor fixes in tutorials 5 and 6.
commit 55a51531be8104f21b3cca3f6bf70b0a7139a041
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 13:51:11 2013 +0100
Updated tutorial 2 for the new project template.
commit 29ddae87bdaecff12dd31b16b000c01efbde9e20
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 11:58:54 2013 +0100
Updated tutorial 1 for the new project template.
commit 0ecb9f6e2514cfd26a678a280d471433375101a3
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 11:29:13 2013 +0100
Adjusted the default URLconf detection to account for the admin.
It's now enabled by default.
commit 5fb4da0d3d09dac28dd94e3fde92b9d4335c0565
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 10:36:55 2013 +0100
Added security warnings for the most sensitive settings.
commit 718d84bd8ac4a42fb4b28ec93965de32680f091e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 23:24:06 2013 +0100
Used an absolute path for the SQLite database.
This ensures the settings file works regardless of which directory
django-admin.py / manage.py is invoked from.
BASE_DIR got a +1 from a BDFL and another core dev. It doesn't involve
the concept of a "Django project"; it's just a convenient way to express
relative paths within the source code repository for non-Python files.
Thanks Jacob Kaplan-Moss for the suggestion.
commit 1b559b4bcda622e10909b68fe5cab90db6727dd9
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 23:22:40 2013 +0100
Removed STATIC_ROOT from the default settings template.
It isn't necessary in development, and it confuses beginners to no end.
Thanks Carl Meyer for the suggestion.
commit a55f141a500bb7c9a1bc259bbe1954c13b199671
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 23:21:43 2013 +0100
Removed MEDIA_ROOT/URL from default settings template.
Many sites will never deal with user-uploaded files, and MEDIA_ROOT is
complicated to explain.
Thanks Carl Meyer for the suggestion.
commit 44bf2f2441420fd9429ee9fe1f7207f92dd87e70
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 22:22:09 2013 +0100
Removed logging config.
This configuration is applied regardless of the value of LOGGING;
duplicating it in LOGGING is confusing.
commit eac747e848eaed65fd5f6f254f0a7559d856f88f
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 22:05:31 2013 +0100
Enabled the locale middleware by default.
USE_I18N is True by default, and doesn't work well without
LocaleMiddleware.
commit d806c62b2d00826dc2688c84b092627b8d571cab
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 22:03:16 2013 +0100
Enabled clickjacking protection by default.
commit 99152c30e6a15003f0b6737dc78e87adf462aacb
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 22:01:48 2013 +0100
Reorganized settings in logical sections, and trimmed comments.
commit d37ffdfcb24b7e0ec7cc113d07190f65fb12fb8a
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:54:11 2013 +0100
Avoided misleading TEMPLATE_DEBUG = DEBUG.
According to the docs TEMPLATE_DEBUG works only when DEBUG = True.
commit 15d9478d3a9850e85841e7cf09cf83050371c6bf
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:46:25 2013 +0100
Removed STATICFILES_FINDERS/TEMPLATE_LOADERS from default settings file.
Only developers with special needs ever need to change these settings.
commit 574da0eb5bfb4570883756914b4dbd7e20e1f61e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:45:01 2013 +0100
Removed STATICFILES/TEMPLATES_DIRS from default settings file.
The current best practice is to put static files and templates in
applications, for easier testing and deployment.
commit 8cb18dbe56629aa1be74718a07e7cc66b4f9c9f0
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:24:16 2013 +0100
Removed settings related to email reporting from default settings file.
While handy for small scale projects, it isn't exactly a best practice.
commit 8ecbfcb3638058f0c49922540f874a7d802d864f
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 18:54:43 2013 +0100
Documented how to enable the sites framework.
commit 23fc91a6fa67d91ddd9d71b1c3e0dc26bdad9841
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:28:59 2013 +0100
Disabled the sites framework by default.
RequestSite does the job for single-domain websites.
commit c4d82eb8afc0eb8568bf9c4d12644272415e3960
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Jan 29 00:08:33 2013 +0100
Added a default admin.py to the application template.
Thanks Ryan D Hiebert for the suggestion.
commit 4071dc771e5c44b1c5ebb9beecefb164ae465e22
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 10:59:49 2013 +0100
Enabled the admin by default.
Everyone uses the admin.
commit c807a31f8d89e7e7fd97380e3023f7983a8b6fcb
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 10:57:05 2013 +0100
Removed admindocs from default project template.
commit 09e4ce0e652a97da1a9e285046a91c8ad7a9189c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:32:52 2013 +0100
Added links to the settings documentation.
commit 5b8f5eaef364eb790fcde6f9e86f7d266074cca8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 11:06:54 2013 +0100
Used a significant example for URLconf includes.
commit 908e91d6fcee2a3cb51ca26ecdf12a6a24e69ef8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 16:22:31 2013 +0100
Moved code comments about WSGI to docs, and rewrote said docs.
commit 50417e51996146f891d08ca8b74dcc736a581932
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Jan 28 15:51:50 2013 +0100
Normalized the default application template.
Removed the default test that 1 + 1 = 2, because it's been committed
way too many times, in too many projects.
Added an import of `render` for views, because the first view will
often be:
def home(request):
return render(request, "mysite/home.html")
They are handled independently now and the latter can be influenced by
the new BaseCommand.leave_locale_alone internal option.
Thanks chrischambers for the report, Claude, lpiatek, neaf and gabooo for
their work on a patch, originally on refs. #17379.
Also:
* Added a ``--no-startup`` option to disable this behavior. Previous
logic to try to execute the code in charge of this funcionality was
flawed (it only tried to do so if the user asked for ipython/bpython
and they weren't found)
* Expand ``~`` in PYTHONSTARTUP value.
Thanks hekevintran at gmail dot com for the report and initial patch.
Refs #3381.
It creates a `locale/django.pot` file once instead of one
`locale/<locale_code>/django.pot` file for every locale involved.
Thanks Michal Čihař for the report and patch.
This allows for a behavior more in line with what is expected by Ipython
users, e.g. the user namespace is initialized from config files, startup
files.
Thanks Benjamin Ragan-Kelley from the IPython dev team for the patch.
This allows for Paginator subclasses to easily override the Page class
that gets used.
Took the opportunity to also do some non-invasive PEP8 tidying of the
paginator module.
Berker Peksag
mlavin
Michael Manfre
Tim Graham
Wojciech Banaś
Aymeric Augustin
José Moreira
Russell Keith-Magee
Andrew Godwin
JustinTArthur
Marc Tamlyn
Claude Paroz
Florian Apolloner
Curtis
Loic Bistuer
Ramiro Morales
Vajrasky Kok
Alex Gaynor
Simon Charette
xuxiang
Bouke Haarsma
Jaap Roes
Milton Mazzarri
coagulant
Boryslav Larin
Anssi Kääriäinen
SusanTan
Alasdair Nicol
Ramiro Morales
Alasdair Nicol
Javed Khan
Unai Zalakain
Larry O'Neill
Tai Lee
Baptiste Mispelon
Helen ST
Kevin Christopher Henry
Markus Holtermann
Gregor MacGregor
John Hensley
homm
Roberto Aguilar
Dan Johnson
Christopher Adams
Carl Meyer
Christopher Medrela
Mel Collins
MinRK
Jose L. Patino
Justin Michalicek
Przemek Lewandowski
Julien Phalip
Benjamin Kagia
ersran9
Tomáš Ehrlich
Wilfred Hughes
Daniel Lindsley
Chris Streeter
René Fleschenberg
Preston Holmes
Andrew Jesaitis
Donald Stufft
Jorge Bastida
Senko Rasic
Łukasz Langa
Jacob Burch
Alex Gaynor
Brendon Crawford
Christoph Sieghart
Matthew Wood
konarkmodi
Tomek Paczkowski
Bas Peschier
Honza Král
Tomasz Rybak
George Song
Marcin Biernat
Erik Romijn
Nick Sandford
Craig Blaszczyk
Anton Baklanov
Chris Beaven
Jannis Leidel