This commit touchs various parts of the code base and test framework. Any
found usage of opening a cursor for the sake of initializing a connection
has been replaced with 'ensure_connection()'.
The purpose of this construct is to test if the django.contrib.sites
application is installed. But in Django 1.9 it will be forbidden to
import the Site model when the django.contrib.sites application isn't
installed.
No model besides Site used this pattern.
Refs #21719, #21923.
Following the app-loading refactor, these objects must live outside of
django.contrib.sites.models because they must be available without
importing the django.contrib.sites.models module when
django.contrib.sites isn't installed.
Refs #21680. Thanks Carl and Loic for reporting this issue.
AppStaticStorage only provided one thing over FileSystemStorage, which was
taking an app name (import path) and translating it into a filesystem
path. This is now something that should be done via app_config.path instead,
leaving AppStaticStorage with no reason for existence. It should be safe to
remove, as it was undocumented internal API.
There was some kind of feature in the AppDirectoriesFinder code related to a
"prefix" attribute on the storage class used by AppDirectoriesFinder. Since
this feature was undocumented, untested, and of unclear purpose, I removed it
as well.
During the admin check for list_editable _check_list_editable_item
should return an empty list if all checks pass. Additionally the
Testcase test_readonly_and_editable was changed to test what the
name implies instead of duplicating the logic of test_readonly.
This is the result of Christopher Medrela's 2013 Summer of Code project.
Thanks also to Preston Holmes, Tim Graham, Anssi Kääriäinen, Florian
Apolloner, and Alex Gaynor for review notes along the way.
Also: Fixes#8579, fixes#3055, fixes#19844.
Allowed users to specify which lookups or transforms ("nested lookus")
are available for fields. The implementation is now class based.
Squashed commit of the following:
commit fa7a7195f1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 18 10:53:24 2014 +0200
Added lookup registration API docs
commit eb1c8ce164
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Tue Jan 14 18:59:36 2014 +0200
Release notes and other minor docs changes
commit 11501c29c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 20:53:03 2014 +0200
Forgot to add custom_lookups tests in prev commit
commit 83173b960e
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:59:12 2014 +0200
Renamed Extract -> Transform
commit 3b18d9f3a1
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:51:53 2014 +0200
Removed suggestion of temporary lookup registration from docs
commit 21d0c7631c
Merge: 2509006f2dc442
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 09:38:23 2014 -0800
Merge pull request #2 from mjtamlyn/lookups_3
Reworked custom lookups docs.
commit f2dc4429a1
Author: Marc Tamlyn <marc.tamlyn@gmail.com>
Date: Sun Jan 12 13:15:05 2014 +0000
Reworked custom lookups docs.
Mostly just formatting and rewording, but also replaced the example
using ``YearExtract`` to use an example which is unlikely to ever be
possible directly in the ORM.
commit 2509006506
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 13:19:13 2014 +0200
Removed unused import
commit 4fba5dfaa0
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:34:41 2014 +0200
Added docs to index
commit 6d53963f37
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:10:24 2014 +0200
Dead code removal
commit f9cc039007
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 19:00:43 2014 +0200
A new try for docs
commit 33aa18a6e3
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:57:12 2014 +0200
Renamed get_cols to get_group_by_cols
commit c7d5f8661b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:45:53 2014 +0200
Altered query string customization for backends vendors
The new way is trying to call first method 'as_' + connection.vendor.
If that doesn't exist, then call as_sql().
Also altered how lookup registration is done. There is now
RegisterLookupMixin class that is used by Field, Extract and
sql.Aggregate. This allows one to register lookups for extracts and
aggregates in the same way lookup registration is done for fields.
commit 90e7004ec1
Merge: 66649fff7c2c0a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:21:01 2014 +0200
Merge branch 'master' into lookups_3
commit 66649ff891
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:16:01 2014 +0200
Some rewording in docs
commit 31b8faa627
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 29 15:52:29 2013 +0200
Cleanup based on review comments
commit 1016159f34
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 18:37:04 2013 +0200
Proof-of-concept fix for #16731
Implemented only for SQLite and PostgreSQL, and only for startswith
and istartswith lookups.
commit 193cd097ca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 17:57:58 2013 +0200
Fixed#11722 -- iexact=F() produced invalid SQL
commit 08ed3c3b49
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:59:52 2013 +0200
Made Lookup and Extract available from django.db.models
commit b99c8d83c9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:06:29 2013 +0200
Fixed review notes by Loic
commit 049eebc070
Merge: ed8fab7b80a835
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:53:10 2013 +0200
Merge branch 'master' into lookups_3
Conflicts:
django/db/models/fields/__init__.py
django/db/models/sql/compiler.py
django/db/models/sql/query.py
tests/null_queries/tests.py
commit ed8fab7fe8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:47:23 2013 +0200
Made Extracts aware of full lookup path
commit 27a57b7aed
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:10:11 2013 +0200
Removed debugger import
commit 074e0f5aca
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:02:16 2013 +0200
GIS lookup support added
commit 760e28e72b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 20:04:31 2013 +0200
Removed usage of Constraint, used Lookup instead
commit eac4776684
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:22:30 2013 +0200
Minor cleanup of Lookup API
commit 2adf50428d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:14:19 2013 +0200
Added documentation, polished implementation
commit 32c04357a8
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:10:15 2013 +0200
Avoid OrderedDict creation on lookup aggregate check
commit 7c8b3a32cc
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:04:34 2013 +0200
Implemented nested lookups
But there is no support of using lookups outside filtering yet.
commit 4d219d4cde
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Wed Nov 27 22:07:30 2013 +0200
Initial implementation of custom lookups
This (nearly) completes the work to isolate all the test modules from
each other. This is now more important as importing models from another
module will case PendingDeprecationWarnings if those modules are not in
INSTALLED_APPS. The only remaining obvious dependencies are:
- d.c.auth depends on d.c.admin (because of the is_admin flag to some
views), but this is not so important and d.c.admin is in
always_installed_apps
- test_client_regress depends on test_client. Eventually these should
become a single module, as the split serves no useful purpose.
Validating STATIC_ROOT in StaticFilesStorage.__init__ turned out to be
problematic - especially with tests - because the storage refuses to work even
if there are no actual interactions with the file system, which is backward
incompatible.
Originally the validation happened in the StaticFilesStorage.path method, but
that didn't work as expected because the call to FileSystemStorage.__init__
replaced the empty value by a valid path. The new approach is to move back the
check to the StaticFilesStorage.path method, but ensure that the location
attribute remains None after the call to super.
Refs #21581.
When STATIC_ROOT wasn't set, collectstatic --clear would delete
every files within the current directory and its descendants.
This patch makes the following changes:
Prevent collectstatic from running if STATIC_ROOT isn't set.
Fixed an issue that prevented collectstatic from displaying the
destination directory.
Changed the warning header to notify when the command is run
in dry-run mode.
Now that the refactorings are complete, it isn't particularly useful any
more, nor very well named. Let's keep the API as simple as possible.
Fixed#21689.
Since it triggers imports, it shouldn't be done lightly.
This commit adds a public API for doing it explicitly, django.setup(),
and does it automatically when using manage.py and wsgi.py.
Thanks Florian for isolating the shortest way to reproduce this issue:
./runtests.py \
django.contrib.auth.tests.test_context_processors.AuthContextProcessorTests.test_perms_attrs \
django.contrib.auth.tests.test_auth_backends.ChangedBackendSettingsTest.test_changed_backend_settings \
django.contrib.auth.tests.test_auth_backends.CustomUserModelBackendAuthenticateTest.test_authenticate \
django.contrib.auth.tests.test_basic.BasicTestCase.test_createsuperuser_management_command
Returning None on errors required unpythonic error checking and was
inconsistent with get_app_config.
get_model was a private API until the previous commit, but given that it
was certainly used in third party software, the change is explained in
the release notes.
Applied the same change to get_registered_model, which is a new private
API introduced during the recent refactoring.
ContentTypes are only created for installed applications, and I could
make a case for not returning a model that isn't installed any more.
The check for stale ContentTypes in update_contenttypes doesn't use
model_class.
ModelSignal actually needs get_registered_model since the lookup happens
at import time. I took this opportunity to perform a small refactoring.
* Introduced [un]set_installed_apps to handle changes to the
INSTALLED_APPS setting.
* Refactored [un]set_available_apps to share its implementation
with [un]set_installed_apps.
* Implemented a receiver to clear some app-related caches.
* Removed test_missing_app as it is basically impossible to reproduce
this situation with public methods of the new app cache.
It was called _populate() before I renamed it to populate(). Since it
has been superseded by populate_models() there's no reason to keep it.
Removed the can_postpone argument of load_app() as it was only used by
populate(). It's a private API and there's no replacement. Simplified
load_app() accordingly. Then new version behaves exactly like the old
one even though it's much shorter.
Since applications that aren't installed no longer have an application
configuration, it is now always True in practice.
Provided an abstraction to temporarily add or remove applications as
several tests messed with app_config.installed to achieve this effect.
For now this API is _-prefixed because it looks dangerous.
* Removed ADMIN_FOR setting and warn warning
* Group view functions by namespace instead of site
* Added a test verifying namespaces are listed
Thanks to Claude Paroz for reviewing and ideas for improvement.
Used the information from the app cache instead of creating a duplicate
based on INSTALLED_APPS.
Model._meta.installed is no longer writable. It was a rather sketchy way
to alter private internals anyway.
Added comments in the three empty models.py files that are still needed.
Adjusted the test runner to add applications corresponding to test
labels to INSTALLED_APPS even when they don't have a models module.
Several parts of Django call get_apps() with a comment along this lines
of "this has the side effect of calling _populate()". I fail to see how
this is better than just calling populate()!
Since the original ones in django.db.models.loading were kept only for
backwards compatibility, there's no need to recreate them. However, many
internals of Django still relied on them.
They were also imported in django.db.models. They never appear in the
documentation, except a quick mention of get_models and get_app in the
1.2 release notes to document an edge case in GIS. I don't think that
makes them a public API.
This commit doesn't change the overall amount of global state but
clarifies that it's tied to the app_cache object instead of hiding it
behind half a dozen functions.
Currently, if the authentication mechanism uses a custom HTTP header
and not REMOTE_USER, it is not easy to test. This commit modifies
remote user tests in order to make them more generic.
Current language is no longer saved to session by LocaleMiddleware
on every response (the behavior introduced in #14825).
Instead language stored in session is reintroduced into new session
after logout.
Forward port of c558a43fd6 to master.
Previously when collecting static files, the directories would receive permissions
from the global umask. Now the default permission comes from FILE_UPLOAD_DIRECTORY_PERMISSIONS
and there's an option to specify the permissions by subclassing any of the
static files storage classes and setting the directory_permissions_mode parameter.
Updated ModelAdmin to use form.instance when passing parent model to
child inlines for add_view. There is effectively no change in the
change_view since the previously passed 'obj' is the same as form.instance.
Thanks to meshy for report, and EvilDMP and timo for review.
Problem were users calling messages.debug/info/* with a wrong argument
and getting the error "You cannot add messages without installing
MessageMiddleware"
Thanks to trac-user merb for the report.
Thanks Curtis Malony and Florian Apolloner.
Squashed commit of the following:
commit 3380495e93
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 14:18:07 2013 +0100
Looked up the template_fragments cache at runtime.
commit 905a74f52b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 14:19:48 2013 +0100
Removed all uses of create_cache.
Refactored the cache tests significantly.
Made it safe to override the CACHES setting.
commit 35e289fe92
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 12:23:57 2013 +0100
Removed create_cache function.
commit 8e274f747a
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 12:04:52 2013 +0100
Updated docs to describe a simplified cache backend API.
commit ee7eb0f73e
Author: Curtis Maloney <curtis@tinbrain.net>
Date: Sat Oct 19 09:49:24 2013 +1100
Fixed#21012 -- Thread-local caches, like databases.
The GenericRelation refactoring removed GenericRelations from
model._meta.many_to_many. This had the side effect of disallowing
editable GenericRelations in ModelForms. Editable GenericRelations
aren't officially supported, but if we don't fix this we don't offer any
upgrade path for those who used the ability to set editable=True
in GenericRelation subclass.
Thanks to Trac alias joshcartme for the report and stephencmd and Loic
for working on this issue.
Unfortunately, escape sequences (\x.. or \u....) do not fit well
with the gettext toolchain. Falling back to using literal char,
even if visibility is not ideal.
Exposing the level name (e.g. "info") makes it possible to prepend
something to the class name. For example, Twitter Bootstrap has
an alert-info class. This class can now be added to the message
using `class="alert-{{ message.level_tag }}".
Because the level_tag was on the end of the `tags` property, it
could not be used in this fashion when extra_tags were given.
Replaced the custom, untested memoize with a similar decorator from Python's
3.2 stdlib. Although some minor performance degradation (see ticket), it is
expected that in the long run lru_cache will outperform memoize once it is
implemented in C.
Thanks to EvilDMP for the report and Baptiste Mispelon for the idea of
replacing memoize with lru_cache.
This patch introduces the Prefetch object which allows customizing prefetch
operations.
This enables things like filtering prefetched relations, calling select_related
from a prefetched relation, or prefetching the same relation multiple times
with different querysets.
When a Prefetch instance specifies a to_attr argument, the result is stored
in a list rather than a QuerySet. This has the fortunate consequence of being
significantly faster. The preformance improvement is due to the fact that we
save the costly creation of a QuerySet instance.
Thanks @akaariai for the original patch and @bmispelon and @timgraham
for the reviews.
``ModelAdmin.view_on_site`` defines wether to show a link to the object on the
admin detail page. If ``True``, cleverness (i.e. ``Model.get_absolute_url``) is
used to get the url. If it's a callable, the callable is called with the object
as the only parameter. If ``False``, not link is displayed.
With the aim of maitaining backwards compatibility, ``True`` is the default.
`HttpRequest.scheme` is `https` if `settings.SECURE_PROXY_SSL_HEADER` is
appropriately set and falls back to `HttpRequest._get_scheme()` (a hook
for subclasses to implement) otherwise.
`WSGIRequest._get_scheme()` makes use of the `wsgi.url_scheme` WSGI
environ variable to determine the request scheme.
`HttpRequest.is_secure()` simply checks if `HttpRequest.scheme` is
`https`.
This provides a way to check the current scheme in templates, for example.
It also allows us to deal with other schemes.
Thanks nslater for the suggestion.
Added a validation error check when creating the permissions for model, to avoid
cryptic database error when the verbose_name is longer than 39 characters
thanks elena for reporting it
The package renaming restores the older package names (which were also the
documented package names). This doesn't affect test discovery because the
module in question doesn't contain any tests.
Thanks to Carl for the design discussion.
Don't set a global default interpreted role function for reStructuredText.
Instead, use the `default-role` directive to change the default only within
the `parse_rst()` function.
Thanks Malcolm Tredinnick for the report.
Squashed commit of the following:
commit 63ddb271a44df389b2c302e421fc17b7f0529755
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 29 22:51:00 2013 +0200
Clarified interactions between atomic and exceptions.
commit 2899ec299228217c876ba3aa4024e523a41c8504
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:45:32 2013 +0200
Fixed TransactionManagementError in tests.
Previous commit introduced an additional check to prevent running
queries in transactions that will be rolled back, which triggered a few
failures in the tests. In practice using transaction.atomic instead of
the low-level savepoint APIs was enough to fix the problems.
commit 4a639b059ea80aeb78f7f160a7d4b9f609b9c238
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Sep 24 22:24:17 2013 +0200
Allowed nesting constraint_checks_disabled inside atomic.
Since MySQL handles transactions loosely, this isn't a problem.
commit 2a4ab1cb6e83391ff7e25d08479e230ca564bfef
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Sep 21 18:43:12 2013 +0200
Prevented running queries in transactions that will be rolled back.
This avoids a counter-intuitive behavior in an edge case on databases
with non-atomic transaction semantics.
It prevents using savepoint_rollback() inside an atomic block without
calling set_rollback(False) first, which is backwards-incompatible in
tests.
Refs #21134.
commit 8e3db393853c7ac64a445b66e57f3620a3fde7b0
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:14:17 2013 +0200
Replaced manual savepoints by atomic blocks.
This ensures the rollback flag is handled consistently in internal APIs.
This reverts commit aae5a96d57.
This fix is no longer necessary, our pbkdf2 (see next commit) implementation
no longer rehashes the password every iteration.
tearDownClass is not called if setUpClass throws an exception, in our case
this means that LiveServerTestCase leaks LiveServerThread sockets if the
test happens to be skipped later on, and AdminSeleniumWebDriverTestCase
doesn't close it's already open browser window. To prevent this leakage
we catch errors where needed and manually call _tearDownClassInternal.
_tearDownClassInternal should be written as defensively as possible since
it is not allowed to make any assumptions on how far setUpClass got.
This patch should fix the sporadic "Address already in use"-errors on jenkins
and also the "This code isn't under transaction management"-error for sqlite
(also just on jenkins).
After discussion with koniiiik, jezdez, kmtracey, tos9, lifeless, nedbat and
voidspace it was decided that this is the safest approach (thanks to everyone
for their comments and help). Manually calling tearDownClass was shut down
cause we don't know how our users override our classes.
This is a private and very specialized API on purpose and should not be used
without a strong reason!
This patch partially reverts the earlier attempts to fix those issues,
namely:
2fa0dd73b1 and
3c5775d36f
Final note: If this patch breaks in a later version of Django, please be
very careful on how you fix it, you might not see test failures locally.
That said, this patch hopefully doesn't produce even more failures.
* Limit the password length to 4096 bytes
* Password hashers will raise a ValueError
* django.contrib.auth forms will fail validation
* Document in release notes that this is a backwards incompatible change
Thanks to Josh Wright for the report, and Donald Stufft for the patch.
This is a security fix; disclosure to follow shortly.
If setUpClass throws an exception (in this case "Address already in use" if the
super call can't find any open port to bind too) tearDownClass is not called.
This results in open browser windows from the webdriver, hence we only construct
it once we are sure there is no code afterwards which could error out.
We want to be able to use it for instance for discovering `tasks.py` modules
inside the INSTALLED_APPS.
This commit therefore moves the logic to `autodiscover_modules` method in
django.utils.module_loading.
The Admin widget, which can be used to filter multiple selects
can sometimes be too narrow and hide information such as
user permissions. This commit adds tooltips to the select
options so that a user can hover over and see the hidden text.
AdminSite now has overridable site_header, site_title and index_title attributes. Changed
each admin view to pass these to the context (in a new AdminSite.each_context() method).
The intent here is to make it easier to override these things in the common case, instead of
having to override a template, which is a bigger burden.
The current auth backend code catches TypeError to detect backends that
do not support specified argumetnts. As a result, any TypeErrors raised
within the actual backend code are silenced.
In Python 2.7+ and 3.2+ this can be avoided by using inspect.getcallargs().
With this method, we can test whether arguments match the signature without
actually calling the function.
Thanks David Eyk for the report.
Added settings.SESSION_SERIALIZER which is the import path of a serializer
to use for sessions.
Thanks apollo13, carljm, shaib, akaariai, charettes, and dstufft for reviews.
The view tried to display links to a ModelAdmin's change_view, which
resulted in NoReverseMatches if get_urls was overridden to remove the
corresponding url.
The default delete action resulted in a NoReverseMatch if it were to
list any Model with a ModelAdmin with `get_urls` overridden to remove
the change url. Catching the error and not displaying the link in that
case, as was already done for models with no registered admins.
Thanks Keryn Knight for the report.
Claude Paroz
Michael Manfre
Aymeric Augustin
Simon Charette
Jannis Leidel
Carl Meyer
Arne Brodowski
Loic Bistuer
Oliver George
Anssi Kääriäinen
Alex Gaynor
Russell Keith-Magee
Tim Graham
Marc Tamlyn
Andrew Godwin
Florian Apolloner
Thomas Schreiber
Jon Lønne
Bouke Haarsma
Bartolomé Sánchez
Jakub Gocławski
Ludwik Trammer
maurycyp
Ramiro Morales
Krzysztof Jurewicz
Vajrasky Kok
Christopher Medrela
Jay Leadbetter
Baptiste Mispelon
Denis Cornehl
Curtis Maloney
glts
Matt Robenolt
Sjoerd Langkemper
Milton Mazzarri
coagulant
Boryslav Larin
Unai Zalakain
Kevin Christopher Henry
Renato Oliveira
Alasdair Nicol
Alasdair Nicol
Kathryn Killebrew
Ryan Allen
Larry O'Neill
joaoxsouls
crccheck
Chris Adams
Étienne Loks
Tai Lee
Ryan Kaskel
Anentropic
tschilling
Adam Haney
Michał Lech
Paul McMillan
Brian Holdefehr
Beshr Kayali
Juan Catalano
Keryn Knight
Gregor MacGregor
Roberto Aguilar
Pablo Mouzo
Julien Phalip
Rudy Mutter
Garry Polley
Adrian Holovaty
Eric Boersma
CHI Cheng
Michał Górny
Erik Romijn
Lukasz Balcerzak
Rainer Koirikivi