File operations always raise a ENOENT error when a file doesn't exist.
Checking the file exists before the operation adds a race condition
condition where the file could be removed between operations. As the
operation already raises an error on a missing file, avoid this race and
avoid checking the file exists twice. Instead only check a file exists
by catching the ENOENT error.
New Storage.get_{accessed,created,modified}_time() methods convert the
naive time from now-deprecated {accessed,created_modified}_time()
methods into aware objects in UTC if USE_TZ=True.
When running collectstatic with a hashing static file storage backend,
URLs referencing other files were normalized with posixpath.normpath.
This could corrupt URLs: for example 'a.css#b/../c' became just 'c'.
Normalization seems to be an artifact of the historical implementation.
It contained a home-grown implementation of posixpath.join which relied
on counting occurrences of .. and /, so multiple / had to be collapsed.
The new implementation introduced in the previous commit doesn't suffer
from this issue. So it seems safe to remove the normalization.
There was a test for this normalization behavior but I don't think it's
a good test. Django shouldn't modify CSS that way. If a developer has
rendundant /s, it's mostly an aesthetic issue and it isn't Django's job
to fix it. Conversely, if the user wants a series of /s, perhaps in the
URL fragment, Django shouldn't destroy it.
Refs #26249.
collectstatic crashed when:
* a hashing static file storage backend was used
* a static file referenced another static file located directly in
STATIC_ROOT (not a subdirectory) with an absolute URL (which must
start with STATIC_URL, which cannot be empty)
It seems to me that the current code reimplements relative path joining
and doesn't handle edge cases correctly. I suspect it assumes that
STATIC_URL is of the form r'/[^/]+/'.
Throwing out that code in favor of the posixpath module makes the logic
easier to follow. Handling absolute paths correctly also becomes easier.
Before cf546e1, static files in form or widget Media were usually
wrapped with contrib.staticfiles.templatetags.staticfiles.static.
This test ensures compatibility with third-party code that's still
using this pattern.
A @font-face declaration may contain a fragment that looks like a relative path,
e.g. @font-face { src: url('../fonts/font.svg#../path/like/fragment'); }
In this case, an incorrect path was passed to the storage backend, which raised
an error that caused collectstatic to crash.
This requires that each test never alters files in static directories
collected by other tests. The alternative is to add a temporary
directory to STATICFILES_DIRS or a new app to INSTALLED_APPS.
Dropped the DJANGO_TEST_TEMP_DIR environment variable.
Before this change, proper removal depended on the developer passing
dir=os.environ['DJANGO_TEST_TMP_DIR'] to tempfile functions.
The stated reason for its introduction in d18d37ce no longer applies
since Django's code repository was switched from Subversion to git.
Furthermore it never had any effect because shutil.rmtree ignores its
onerror argument when ignore_errors is True.
The reason for its use in template management commands is unclear.
When using ManifestStaticFilesStorage, deleted static files would be
correctly cleaned up by "collectstatic --clear", but the manifest file
would still contain the stale entries.
Thanks to tedtieken for the report
Before 4befb30 the detection was broken because we used isinstance
against a LazyObject rather than against a Storage class. That commit
fixed it by looking directly at the object wrapped by LazyObject.
This could however be a problem to anyone who subclasses the
collectstatic management Command and directly supplies a Storage class.
Refs #21581.
Added searched_locations in finders module. Added verbosity flag level 2 on 'findstatic'
command that will output the directories on which it searched the relative paths.
Reported by ccurvey. Initial patch by Jonas Svensson and Vajrasky Kok.
AppStaticStorage only provided one thing over FileSystemStorage, which was
taking an app name (import path) and translating it into a filesystem
path. This is now something that should be done via app_config.path instead,
leaving AppStaticStorage with no reason for existence. It should be safe to
remove, as it was undocumented internal API.
There was some kind of feature in the AppDirectoriesFinder code related to a
"prefix" attribute on the storage class used by AppDirectoriesFinder. Since
this feature was undocumented, untested, and of unclear purpose, I removed it
as well.
When STATIC_ROOT wasn't set, collectstatic --clear would delete
every files within the current directory and its descendants.
This patch makes the following changes:
Prevent collectstatic from running if STATIC_ROOT isn't set.
Fixed an issue that prevented collectstatic from displaying the
destination directory.
Changed the warning header to notify when the command is run
in dry-run mode.
Added comments in the three empty models.py files that are still needed.
Adjusted the test runner to add applications corresponding to test
labels to INSTALLED_APPS even when they don't have a models module.
Previously when collecting static files, the directories would receive permissions
from the global umask. Now the default permission comes from FILE_UPLOAD_DIRECTORY_PERMISSIONS
and there's an option to specify the permissions by subclassing any of the
static files storage classes and setting the directory_permissions_mode parameter.
Replaced the custom, untested memoize with a similar decorator from Python's
3.2 stdlib. Although some minor performance degradation (see ticket), it is
expected that in the long run lru_cache will outperform memoize once it is
implemented in C.
Thanks to EvilDMP for the report and Baptiste Mispelon for the idea of
replacing memoize with lru_cache.
Previously, when collecting static files, the files would receive permission
from FILE_UPLOAD_PERMISSIONS. Now, there's an option to give different
permission from uploaded files permission by subclassing any of the static
files storage classes and setting the file_permissions_mode parameter.
Thanks dblack at atlassian.com for the suggestion.
Jon Dufresne
Claude Paroz
Berker Peksag
James Aylett
Aymeric Augustin
Hasan
Johannes Hoppe
Josh Soref
Alex Morozov
msaelices
Yusuke Miyazaki
Andriy Sokolovskiy
Moritz Sichert
Markus Amalthea Magnuson
Simon Charette
Tim Graham
Sztrovacsek
Aymeric Augustin
mlavin
darkryder
Diego Guimarães
Trey Hunner
Denis Cornehl
Huu Nguyen
Florian Apolloner
Anubhav Joshi
David Evans
Loic Bistuer
Jannis Leidel
Vajrasky Kok
Carl Meyer
Bouke Haarsma
Jason Myers
Ramiro Morales
Tai Lee
Florian Hahn