Commit Graph

59 Commits

Author SHA1 Message Date
Claude Paroz edcecaf0de Fixed #19670 -- Applied CachedFilesMixin patterns to specific extensions
Thanks Simon Meers for the initial patch, and Tim Graham for the review.
2016-03-30 14:34:41 +02:00
Aymeric Augustin 7f6fbc906a Prevented static file corruption when URL fragment contains '..'.
When running collectstatic with a hashing static file storage backend,
URLs referencing other files were normalized with posixpath.normpath.
This could corrupt URLs: for example 'a.css#b/../c' became just 'c'.

Normalization seems to be an artifact of the historical implementation.
It contained a home-grown implementation of posixpath.join which relied
on counting occurrences of .. and /, so multiple / had to be collapsed.

The new implementation introduced in the previous commit doesn't suffer
from this issue. So it seems safe to remove the normalization.

There was a test for this normalization behavior but I don't think it's
a good test. Django shouldn't modify CSS that way. If a developer has
rendundant /s, it's mostly an aesthetic issue and it isn't Django's job
to fix it. Conversely, if the user wants a series of /s, perhaps in the
URL fragment, Django shouldn't destroy it.

Refs #26249.
2016-02-23 19:35:16 +01:00
Aymeric Augustin 706b33fef8 Fixed #26249 -- Fixed collectstatic crash for files in STATIC_ROOT referenced by absolute URL.
collectstatic crashed when:

* a hashing static file storage backend was used
* a static file referenced another static file located directly in
  STATIC_ROOT (not a subdirectory) with an absolute URL (which must
  start with STATIC_URL, which cannot be empty)

It seems to me that the current code reimplements relative path joining
and doesn't handle edge cases correctly. I suspect it assumes that
STATIC_URL is of the form r'/[^/]+/'.

Throwing out that code in favor of the posixpath module makes the logic
easier to follow. Handling absolute paths correctly also becomes easier.
2016-02-23 19:34:21 +01:00
userimack 60586dd737 Fixed #26125 -- Fixed E731 flake warnings. 2016-01-25 14:23:43 -05:00
msaelices 7624fdb9f8 Fixed #25283 -- Fixed collectstatic crash if a URL contains a fragment with a path.
A @font-face declaration may contain a fragment that looks like a relative path,
e.g. @font-face { src: url('../fonts/font.svg#../path/like/fragment'); }
In this case, an incorrect path was passed to the storage backend, which raised
an error that caused collectstatic to crash.
2015-11-12 19:30:48 -05:00
Piotr Jakimiak 4157c502a5 Removed unnecessary arguments in .get method calls 2015-05-13 20:51:18 +02:00
Tim Graham 0ed7d15563 Sorted imports with isort; refs #23860. 2015-02-06 08:16:28 -05:00
Tim Graham 0af4ddabf4 Fixed typos in comments. 2014-10-08 15:59:19 -04:00
Denis Cornehl 3bec38888f Fixed #22557 -- ManifestStaticFilesStorage did not cleanup deleted files.
When using ManifestStaticFilesStorage, deleted static files would be
correctly cleaned up by "collectstatic --clear", but the manifest file
would still contain the stale entries.

Thanks to tedtieken for the report
2014-05-20 18:14:49 +02:00
David Evans 86dcac4634 Fixed #22315 -- str/bytes mismatch in staticfiles
Previously, `ManifestFilesMixin.read_manifest` failed in Python 3
because `json.loads` accepts `str` not `bytes`.
2014-03-25 09:53:24 -04:00
Carl Meyer f56c88a8ee Fixed #21867 -- Removed AppStaticStorage; app paths are now AppConfig's job.
AppStaticStorage only provided one thing over FileSystemStorage, which was
taking an app name (import path) and translating it into a filesystem
path. This is now something that should be done via app_config.path instead,
leaving AppStaticStorage with no reason for existence. It should be safe to
remove, as it was undocumented internal API.

There was some kind of feature in the AppDirectoriesFinder code related to a
"prefix" attribute on the storage class used by AppDirectoriesFinder. Since
this feature was undocumented, untested, and of unclear purpose, I removed it
as well.
2014-01-24 15:32:03 -07:00
Jannis Leidel 8efd20f96d Added ManifestStaticFilesStorage to staticfiles contrib app.
It uses a static manifest file that is created when running
collectstatic in the JSON format.
2014-01-20 19:32:07 +01:00
Loic Bistuer 1e9e7351f8 Fixed #21750 -- Fixed regression introduced by 4befb30.
Validating STATIC_ROOT in StaticFilesStorage.__init__ turned out to be
problematic - especially with tests - because the storage refuses to work even
if there are no actual interactions with the file system, which is backward
incompatible.

Originally the validation happened in the StaticFilesStorage.path method, but
that didn't work as expected because the call to FileSystemStorage.__init__
replaced the empty value by a valid path. The new approach is to move back the
check to the StaticFilesStorage.path method, but ensure that the location
attribute remains None after the call to super.

Refs #21581.
2014-01-11 08:20:28 -05:00
Loic Bistuer 4befb3015c Fixed #21581 -- Fixed a number of issues with collectstatic.
When STATIC_ROOT wasn't set, collectstatic --clear would delete
every files within the current directory and its descendants.

This patch makes the following changes:

Prevent collectstatic from running if STATIC_ROOT isn't set.

Fixed an issue that prevented collectstatic from displaying the
destination directory.

Changed the warning header to notify when the command is run
in dry-run mode.
2013-12-31 14:58:49 -05:00
Curtis Maloney ffc37e2343 Fixed #21012 -- New API to access cache backends.
Thanks Curtis Malony and Florian Apolloner.

Squashed commit of the following:

commit 3380495e93
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sat Nov 23 14:18:07 2013 +0100

    Looked up the template_fragments cache at runtime.

commit 905a74f52b
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sat Nov 23 14:19:48 2013 +0100

    Removed all uses of create_cache.

    Refactored the cache tests significantly.

    Made it safe to override the CACHES setting.

commit 35e289fe92
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sat Nov 23 12:23:57 2013 +0100

    Removed create_cache function.

commit 8e274f747a
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date:   Sat Nov 23 12:04:52 2013 +0100

    Updated docs to describe a simplified cache backend API.

commit ee7eb0f73e
Author: Curtis Maloney <curtis@tinbrain.net>
Date:   Sat Oct 19 09:49:24 2013 +1100

    Fixed #21012 -- Thread-local caches, like databases.
2013-11-23 15:06:59 +01:00
Aymeric Augustin 6a6428a36f Took advantage of django.utils.six.moves.urllib.*. 2013-09-05 14:39:23 -05:00
Curtis Maloney 07876cf02b Deprecated SortedDict (replaced with collections.OrderedDict)
Thanks Loic Bistuer for the review.
2013-08-04 07:09:39 -04:00
Claude Paroz fdd7a355bf Deprecated django.utils.importlib
This was a shim for pre-Python 2.7 support.
2013-07-29 17:10:22 +02:00
Tim Graham 9b88dd3809 Fixed #20679 -- Corrected CachedFilesMixin.post_process docstring.
Thanks bmispelon for the report.
2013-07-25 12:31:53 -04:00
Aymeric Augustin 90fe9141de Fixed #18986 -- Improved error message for missing files
in CachedStaticFilesStorage. Thanks zyegfryed for his work on the patch.
2013-04-28 16:45:05 +02:00
Aleksandra Sendecka 32a7ab2148 Fixed #19526
CSS specifications governs that syntax is case insensitive.
This modifies CachedFilesMixin to support that.
2013-02-23 23:05:22 +01:00
Claude Paroz c91667338a Fixed #19357 -- Allow non-ASCII chars in filesystem paths
Thanks kujiu for the report and Aymeric Augustin for the review.
2012-12-08 11:13:52 +01:00
Claude Paroz ebc773ada3 Replaced many smart_bytes by force_bytes
In all those occurrences, we didn't care about preserving the
lazy status of the strings, but we really wanted to obtain a
real bytestring.
2012-08-29 11:20:32 +02:00
Alex Gaynor d674bd603e Final explicit closing for staticfiles, they now pass on python3 with -Wall and there are no warnings about unclosed files 2012-08-15 01:29:05 -07:00
Claude Paroz 7d48e077b5 [py3] Fixed staticfiles tests 2012-08-14 17:24:31 +02:00
Karen Tracey b82eb10b26 Fixed #18754 -- cache keys created by post_process
Corrected to always generate the cache keys from file names with
forward slashes, not backslashes.
2012-08-11 17:05:26 -04:00
Aymeric Augustin c5ef65bcf3 [py3] Ported django.utils.encoding.
* Renamed smart_unicode to smart_text (but kept the old name under
  Python 2 for backwards compatibility).
* Renamed smart_str to smart_bytes.
* Re-introduced smart_str as an alias for smart_text under Python 3
  and smart_bytes under Python 2 (which is backwards compatible).
  Thus smart_str always returns a str objects.
* Used the new smart_str in a few places where both Python 2 and 3
  want a str.
2012-08-07 12:00:22 +02:00
Aymeric Augustin 0d914d08a0 [py3] Updated urllib/urllib2/urlparse imports.
Lots of functions were moved. Use explicit imports in all cases
to keey it easy to identify where the functions come from.
2012-07-22 09:29:55 +02:00
Jannis Leidel 1aa0d8ac4d Fixed #18487 -- Made sure that protocol-relative URLs aren't processed by the cached staticfiles storage. Thanks to LukaszBalcerzak for the patch. 2012-07-08 18:25:12 +02:00
Jannis Leidel 3047981517 Fixed #18050 -- Fixed a rather glaring bug in the handling of @import statements when using the cached staticfiles storage. 2012-07-08 18:17:53 +02:00
Jannis Leidel 3727f6d096 Fixed #18430 -- Use the FILE_CHARSET setting when reading from a file during post processing with the cached staticfiles storage. Thanks to Brant Young for initial debugging. 2012-07-08 12:56:49 +02:00
Claude Paroz 4a103086d5 Fixed #18269 -- Applied unicode_literals for Python 3 compatibility.
Thanks Vinay Sajip for the support of his django3 branch and
Jannis Leidel for the review.
2012-06-07 18:08:47 +02:00
Jannis Leidel 5f75ac91df Fixed #17896 -- Added file_hash method to CachedStaticFilesStorage to be able to customize the way the hashed name of a file is created. Thanks to mkai for the initial patch. 2012-05-16 13:21:50 +02:00
Jannis Leidel 1c1a229632 Set the post process cache when finished instead of one by one.
This should prevent a race condition if running collectstatic is
canceled or its cache is accessed from other processes, leaving the
cache in a corrupt state.
2012-05-10 18:31:48 +02:00
Claude Paroz 9383a2761c Removed with_statement imports, useless in Python >= 2.6. Refs #17965. Thanks jonash for the patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17828 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-03-30 08:02:08 +00:00
Jannis Leidel 0ee801e360 Fixed #17857 -- Stopped CachedStaticFilesStorage from creating absolute URLs unnecessarily. Thanks, tgecho.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17697 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-03-13 03:48:11 +00:00
Jannis Leidel 7487c74020 Fixed #17865 -- Strip whitespaces from the paths when using the CachedStaticFilesStorage backend.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17696 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-03-13 03:48:03 +00:00
Jannis Leidel 3a5f9cd1ae Fixed #17861 -- Took care of special characters when creating the staticfiles storage cache keys. Many thanks to Preston Holmes.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17688 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-03-12 20:28:09 +00:00
Jannis Leidel e88a65660f Fixed #17806 -- Fixed off-by-n error in the CachedStaticFilesStorage that prevented it from finding files in nested directories. Many thanks to lpetre.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17622 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-03-02 11:29:14 +00:00
Jannis Leidel 758a0cd0f1 Fixed #17689 -- Stopped the CachedStaticFilesStorage from trying to hash paths that aren't files.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17535 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-02-16 14:40:32 +00:00
Jannis Leidel 803de60c08 Stopped the collectstatic management from being wastful with file handlers. Refs r17519.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17521 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-02-14 09:55:09 +00:00
Jannis Leidel 4f1ac8f5f1 Minor bugfixing of the staticfiles app following upstream development in django-staticfiles.
- Create the files to ignore during the tests dynamically (.hidden and backup~)
- Refactored the post_processing method of the CachedFilesMixin storage mixin to be less time consuming.
- Refactored handling of fragments in the post_process method.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@17519 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2012-02-13 10:51:17 +00:00
Jannis Leidel 46c12d1293 Fixed #17455 -- Extended `CachedStaticFilesStorage` slightly to handle some URLs better that are used to add support for webfonts to IE 6-8. Also ignore `data:` URLs and fragment-only URLs (e.g. `#default#VML`).
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17282 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-12-27 22:49:24 +00:00
Karen Tracey b045783729 Fixed #17462: Adjusted staticfiles CachedFilesMixin code so its tests pass on Windows.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17276 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-12-25 19:44:20 +00:00
Jannis Leidel a82488bf4b Fixed #16966 -- Stopped CachedStaticFilesStorage from choking on querystrings and path fragments.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17068 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-11-03 10:51:02 +00:00
Jannis Leidel 0e2c543979 Fixed #16967 -- Made sure CachedStaticFilesStorage repopulates its cache if there was a miss (for example if the cache server went down).
git-svn-id: http://code.djangoproject.com/svn/django/trunk@17067 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-11-02 17:45:32 +00:00
Jannis Leidel eb5df8e98d Fixed the relative static file resolution of the CachedStaticFilesStorage backend and the post processing of deeply nested static files.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16862 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-09-21 15:58:21 +00:00
Jannis Leidel 2980f2ab9a Fixed #16629 -- Relaxed check for STATIC_ROOT and STATIC_URL settings slightly to only raise an exception if really needed.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16617 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-14 18:15:04 +00:00
Ramiro Morales 21705bd97a Added with_statement Python 2.5 compatibility import missed in [16594]. Thanks Julien.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@16596 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-12 00:29:34 +00:00
Jannis Leidel 1d32bdd3c9 Fixed #15252 -- Added static template tag and CachedStaticFilesStorage to staticfiles contrib app.
Many thanks to Florian Apolloner and Jacob Kaplan-Moss for reviewing and eagle eyeing.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@16594 bcc190cf-cafb-0310-a4f2-bffc1f526a37
2011-08-11 14:07:39 +00:00