p31829507
/
django
mirror of https://github.com/django/django.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[1.7.x] Fixed #9535 -- Added a reference guide for file upload classes. 

Backport of c8c2b8a638 from master
This commit is contained in:
Anubhav Joshi 2014-03-26 19:00:47 +05:30 committed by Tim Graham
parent 61b7fcf478
commit d9c7d9eb6e
4 changed files with 322 additions and 322 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/ref/files/index.txt
View File

@ -10,3 +10,4 @@ File handling
file
storage
uploads

244
docs/ref/files/uploads.txt Normal file
View File

@ -0,0 +1,244 @@
==================================
Uploaded Files and Upload Handlers
==================================
.. module:: django.core.files.uploadedfile
:synopsis: Classes representing uploaded files.
Uploaded files
==============
.. class:: UploadedFile
During file uploads, the actual file data is stored in :attr:`request.FILES
<django.http.HttpRequest.FILES>`. Each entry in this dictionary is an
``UploadedFile`` object (or a subclass) -- a simple wrapper around an uploaded
file. You'll usually use one of these methods to access the uploaded content:
.. method:: UploadedFile.read()
Read the entire uploaded data from the file. Be careful with this method:
if the uploaded file is huge it can overwhelm your system if you try to
read it into memory. You'll probably want to use ``chunks()`` instead; see
below.
.. method:: UploadedFile.multiple_chunks(chunk_size=None)
Returns ``True`` if the uploaded file is big enough to require reading in
multiple chunks. By default this will be any file larger than 2.5 megabytes,
but that's configurable; see below.
.. method:: UploadedFile.chunks(chunk_size=None)
A generator returning chunks of the file. If ``multiple_chunks()`` is
``True``, you should use this method in a loop instead of ``read()``.
In practice, it's often easiest simply to use ``chunks()`` all the time.
Looping over ``chunks()`` instead of using ``read()`` ensures that large
files don't overwhelm your system's memory.
Here are some useful attributes of ``UploadedFile``:
.. attribute:: UploadedFile.name
The name of the uploaded file (e.g. ``my_file.txt``).
.. attribute:: UploadedFile.size
The size, in bytes, of the uploaded file.
.. attribute:: UploadedFile.content_type
The content-type header uploaded with the file (e.g. :mimetype:`text/plain`
or :mimetype:`application/pdf`). Like any data supplied by the user, you
shouldn't trust that the uploaded file is actually this type. You'll still
need to validate that the file contains the content that the content-type
header claims -- "trust but verify."
.. attribute:: UploadedFile.content_type_extra
.. versionadded:: 1.7
A dictionary containing extra parameters passed to the ``content-type``
header. This is typically provided by services, such as Google App Engine,
that intercept and handle file uploads on your behalf. As a result your
handler may not receive the uploaded file content, but instead a URL or
other pointer to the file. (see `RFC 2388`_ section 5.3).
.. _RFC 2388: http://www.ietf.org/rfc/rfc2388.txt
.. attribute:: UploadedFile.charset
For :mimetype:`text/*` content-types, the character set (i.e. ``utf8``)
supplied by the browser. Again, "trust but verify" is the best policy here.
.. note::
Like regular Python files, you can read the file line-by-line simply by
iterating over the uploaded file:
.. code-block:: python
for line in uploadedfile:
do_something_with(line)
However, *unlike* standard Python files, :class:`UploadedFile` only
understands ``\n`` (also known as "Unix-style") line endings. If you know
that you need to handle uploaded files with different line endings, you'll
need to do so in your view.
Subclasses of ``UploadedFile`` include:
.. class:: TemporaryUploadedFile
A file uploaded to a temporary location (i.e. stream-to-disk). This class
is used by the
:class:`~django.core.files.uploadhandler.TemporaryFileUploadHandler`. In
addition to the methods from :class:`UploadedFile`, it has one additional
method:
.. method:: TemporaryUploadedFile.temporary_file_path()
Returns the full path to the temporary uploaded file.
.. class:: InMemoryUploadedFile
A file uploaded into memory (i.e. stream-to-memory). This class is used
by the :class:`~django.core.files.uploadhandler.MemoryFileUploadHandler`.
Built-in upload handers
=======================
.. module:: django.core.files.uploadhandler
:synopsis: Django's handlers for file uploads.
Together the :class:`MemoryFileUploadHandler` and
:class:`TemporaryFileUploadHandler` provide Django's default file upload
behavior of reading small files into memory and large ones onto disk. They
are located in ``django.core.files.uploadhandler``.
.. class:: MemoryFileUploadHandler
File upload handler to stream uploads into memory (used for small files).
.. class:: TemporaryFileUploadHandler
Upload handler that streams data into a temporary file using
:class:`~django.core.files.uploadedfile.TemporaryUploadedFile`.
.. _custom_upload_handlers:
Writing custom upload handlers
==============================
.. class:: FileUploadHandler
All file upload handlers should be subclasses of
``django.core.files.uploadhandler.FileUploadHandler``. You can define upload
handlers wherever you wish.
Required methods
~~~~~~~~~~~~~~~~
Custom file upload handlers **must** define the following methods:
.. method:: FileUploadHandler.receive_data_chunk(raw_data, start)
Receives a "chunk" of data from the file upload.
``raw_data`` is a byte string containing the uploaded data.
``start`` is the position in the file where this ``raw_data`` chunk
begins.
The data you return will get fed into the subsequent upload handlers'
``receive_data_chunk`` methods. In this way, one handler can be a
"filter" for other handlers.
Return ``None`` from ``receive_data_chunk`` to short-circuit remaining
upload handlers from getting this chunk. This is useful if you're
storing the uploaded data yourself and don't want future handlers to
store a copy of the data.
If you raise a ``StopUpload`` or a ``SkipFile`` exception, the upload
will abort or the file will be completely skipped.
.. method:: FileUploadHandler.file_complete(file_size)
Called when a file has finished uploading.
The handler should return an ``UploadedFile`` object that will be stored
in ``request.FILES``. Handlers may also return ``None`` to indicate that
the ``UploadedFile`` object should come from subsequent upload handlers.
Optional methods
~~~~~~~~~~~~~~~~
Custom upload handlers may also define any of the following optional methods or
attributes:
.. attribute:: FileUploadHandler.chunk_size
Size, in bytes, of the "chunks" Django should store into memory and feed
into the handler. That is, this attribute controls the size of chunks
fed into ``FileUploadHandler.receive_data_chunk``.
For maximum performance the chunk sizes should be divisible by ``4`` and
should not exceed 2 GB (2\ :sup:`31` bytes) in size. When there are
multiple chunk sizes provided by multiple handlers, Django will use the
smallest chunk size defined by any handler.
The default is 64*2\ :sup:`10` bytes, or 64 KB.
.. method:: FileUploadHandler.new_file(field_name, file_name, content_type, content_length, charset, content_type_extra)
Callback signaling that a new file upload is starting. This is called
before any data has been fed to any upload handlers.
``field_name`` is a string name of the file ``<input>`` field.
``file_name`` is the unicode filename that was provided by the browser.
``content_type`` is the MIME type provided by the browser -- E.g.
``'image/jpeg'``.
``content_length`` is the length of the image given by the browser.
Sometimes this won't be provided and will be ``None``.
``charset`` is the character set (i.e. ``utf8``) given by the browser.
Like ``content_length``, this sometimes won't be provided.
``content_type_extra`` is extra information about the file from the
``content-type`` header. See :attr:`UploadedFile.content_type_extra
<django.core.files.uploadedfile.UploadedFile.content_type_extra>`.
This method may raise a ``StopFutureHandlers`` exception to prevent
future handlers from handling this file.
.. versionadded:: 1.7
The ``content_type_extra`` parameter was added.
.. method:: FileUploadHandler.upload_complete()
Callback signaling that the entire upload (all files) has completed.
.. method:: FileUploadHandler.handle_raw_input(input_data, META, content_length, boundary, encoding)
Allows the handler to completely override the parsing of the raw
HTTP input.
``input_data`` is a file-like object that supports ``read()``-ing.
``META`` is the same object as ``request.META``.
``content_length`` is the length of the data in ``input_data``. Don't
read more than ``content_length`` bytes from ``input_data``.
``boundary`` is the MIME boundary for this request.
``encoding`` is the encoding of the request.
Return ``None`` if you want upload handling to continue, or a tuple of
``(POST, FILES)`` if you want to return the new data structures suitable
for the request directly.

32
docs/ref/request-response.txt
View File

@ -132,7 +132,7 @@ All attributes should be considered read-only, unless stated otherwise below.
A dictionary-like object containing all uploaded files. Each key in
``FILES`` is the ``name`` from the ``<input type="file" name="" />``. Each
value in ``FILES`` is an :class:`UploadedFile` as described below.
value in ``FILES`` is an :class:`~django.core.files.uploadedfile.UploadedFile`.
See :doc:`/topics/files` for more information.
@ -334,36 +334,6 @@ Methods
process(element)
UploadedFile objects
====================
.. class:: UploadedFile
Attributes
----------
.. attribute:: UploadedFile.name
The name of the uploaded file.
.. attribute:: UploadedFile.size
The size, in bytes, of the uploaded file.
Methods
----------
.. method:: UploadedFile.chunks(chunk_size=None)
Returns a generator that yields sequential chunks of data.
.. method:: UploadedFile.read(num_bytes=None)
Read a number of bytes from the file.
QueryDict objects
=================

367
docs/topics/http/file-uploads.txt
View File

@ -64,49 +64,7 @@ something like::
Notice that we have to pass :attr:`request.FILES <django.http.HttpRequest.FILES>`
into the form's constructor; this is how file data gets bound into a form.
Handling uploaded files
-----------------------
.. class:: UploadedFile
The final piece of the puzzle is handling the actual file data from
:attr:`request.FILES <django.http.HttpRequest.FILES>`. Each entry in this
dictionary is an ``UploadedFile`` object -- a simple wrapper around an uploaded
file. You'll usually use one of these methods to access the uploaded content:
.. method:: read()
Read the entire uploaded data from the file. Be careful with this
method: if the uploaded file is huge it can overwhelm your system if you
try to read it into memory. You'll probably want to use ``chunks()``
instead; see below.
.. method:: multiple_chunks()
Returns ``True`` if the uploaded file is big enough to require
reading in multiple chunks. By default this will be any file
larger than 2.5 megabytes, but that's configurable; see below.
.. method:: chunks()
A generator returning chunks of the file. If ``multiple_chunks()`` is
``True``, you should use this method in a loop instead of ``read()``.
In practice, it's often easiest simply to use ``chunks()`` all the time;
see the example below.
.. attribute:: name
The name of the uploaded file (e.g. ``my_file.txt``).
.. attribute:: size
The size, in bytes, of the uploaded file.
There are a few other methods and attributes available on ``UploadedFile``
objects; see `UploadedFile objects`_ for a complete reference.
Putting it all together, here's a common way you might handle an uploaded file::
Here's a common way you might handle an uploaded file::
def handle_uploaded_file(f):
with open('some/file/name.txt', 'wb+') as destination:
@ -116,6 +74,79 @@ Putting it all together, here's a common way you might handle an uploaded file::
Looping over ``UploadedFile.chunks()`` instead of using ``read()`` ensures that
large files don't overwhelm your system's memory.
There are a few other methods and attributes available on ``UploadedFile``
objects; see :class:`UploadedFile` for a complete reference.
Handling uploaded files with a model
------------------------------------
If you're saving a file on a :class:`~django.db.models.Model` with a
:class:`~django.db.models.FileField`, using a :class:`~django.forms.ModelForm`
makes this process much easier. The file object will be saved to the location
specified by the :attr:`~django.db.models.FileField.upload_to` argument of the
corresponding :class:`~django.db.models.FileField` when calling
``form.save()``::
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import ModelFormWithFileField
def upload_file(request):
if request.method == 'POST':
form = ModelFormWithFileField(request.POST, request.FILES)
if form.is_valid():
# file is saved
form.save()
return HttpResponseRedirect('/success/url/')
else:
form = ModelFormWithFileField()
return render(request, 'upload.html', {'form': form})
If you are constructing an object manually, you can simply assign the file
object from :attr:`request.FILES <django.http.HttpRequest.FILES>` to the file
field in the model::
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import UploadFileForm
from .models import ModelWithFileField
def upload_file(request):
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
instance = ModelWithFileField(file_field=request.FILES['file'])
instance.save()
return HttpResponseRedirect('/success/url/')
else:
form = UploadFileForm()
return render(request, 'upload.html', {'form': form})
Upload Handlers
===============
.. currentmodule:: django.core.files.uploadhandler
When a user uploads a file, Django passes off the file data to an *upload
handler* -- a small class that handles file data as it gets uploaded. Upload
handlers are initially defined in the :setting:`FILE_UPLOAD_HANDLERS` setting,
which defaults to::
("django.core.files.uploadhandler.MemoryFileUploadHandler",
"django.core.files.uploadhandler.TemporaryFileUploadHandler",)
Together :class:`MemoryFileUploadHandler` and
:class:`TemporaryFileUploadHandler` provide Django's default file upload
behavior of reading small files into memory and large ones onto disk.
You can write custom handlers that customize how Django handles files. You
could, for example, use custom handlers to enforce user-level quotas, compress
data on the fly, render progress bars, and even send data to another storage
location directly without storing it locally. See :ref:`custom_upload_handlers`
for details on how you can customize or completely replace upload behavior.
.. _modifying_upload_handlers_on_the_fly:
Where uploaded data is stored
-----------------------------
@ -132,8 +163,7 @@ like ``/tmp/tmpzfp6I6.upload``. If an upload is large enough, you can watch this
file grow in size as Django streams the data onto disk.
These specifics -- 2.5 megabytes; ``/tmp``; etc. -- are simply "reasonable
defaults". Read on for details on how you can customize or completely replace
upload behavior.
defaults" which can be customized as described in the next section.
Changing upload handler behavior
--------------------------------
@ -184,134 +214,7 @@ There are a few settings which control Django's file upload behavior:
:setting:`FILE_UPLOAD_HANDLERS`
The actual handlers for uploaded files. Changing this setting allows
complete customization -- even replacement -- of Django's upload
process. See `upload handlers`_, below, for details.
Defaults to::
("django.core.files.uploadhandler.MemoryFileUploadHandler",
"django.core.files.uploadhandler.TemporaryFileUploadHandler",)
Which means "try to upload to memory first, then fall back to temporary
files."
Handling uploaded files with a model
------------------------------------
If you're saving a file on a :class:`~django.db.models.Model` with a
:class:`~django.db.models.FileField`, using a :class:`~django.forms.ModelForm`
makes this process much easier. The file object will be saved to the location
specified by the :attr:`~django.db.models.FileField.upload_to` argument of the
corresponding :class:`~django.db.models.FileField` when calling
``form.save()``::
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import ModelFormWithFileField
def upload_file(request):
if request.method == 'POST':
form = ModelFormWithFileField(request.POST, request.FILES)
if form.is_valid():
# file is saved
form.save()
return HttpResponseRedirect('/success/url/')
else:
form = ModelFormWithFileField()
return render(request, 'upload.html', {'form': form})
If you are constructing an object manually, you can simply assign the file
object from :attr:`request.FILES <django.http.HttpRequest.FILES>` to the file
field in the model::
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import UploadFileForm
from .models import ModelWithFileField
def upload_file(request):
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
instance = ModelWithFileField(file_field=request.FILES['file'])
instance.save()
return HttpResponseRedirect('/success/url/')
else:
form = UploadFileForm()
return render(request, 'upload.html', {'form': form})
``UploadedFile`` objects
========================
In addition to those inherited from :class:`~django.core.files.File`, all
``UploadedFile`` objects define the following methods/attributes:
.. attribute:: UploadedFile.content_type
The content-type header uploaded with the file (e.g. :mimetype:`text/plain`
or :mimetype:`application/pdf`). Like any data supplied by the user, you
shouldn't trust that the uploaded file is actually this type. You'll still
need to validate that the file contains the content that the content-type
header claims -- "trust but verify."
.. attribute:: UploadedFile.content_type_extra
.. versionadded:: 1.7
A dictionary containing extra parameters passed to the ``content-type``
header. This is typically provided by services, such as Google App Engine,
that intercept and handle file uploads on your behalf. As a result your
handler may not receive the uploaded file content, but instead a URL or
other pointer to the file. (see `RFC 2388`_ section 5.3).
.. _RFC 2388: http://www.ietf.org/rfc/rfc2388.txt
.. attribute:: UploadedFile.charset
For :mimetype:`text/*` content-types, the character set (i.e. ``utf8``)
supplied by the browser. Again, "trust but verify" is the best policy here.
.. attribute:: UploadedFile.temporary_file_path()
Only files uploaded onto disk will have this method; it returns the full
path to the temporary uploaded file.
.. note::
Like regular Python files, you can read the file line-by-line simply by
iterating over the uploaded file:
.. code-block:: python
for line in uploadedfile:
do_something_with(line)
However, *unlike* standard Python files, :class:`UploadedFile` only
understands ``\n`` (also known as "Unix-style") line endings. If you know
that you need to handle uploaded files with different line endings, you'll
need to do so in your view.
Upload Handlers
===============
When a user uploads a file, Django passes off the file data to an *upload
handler* -- a small class that handles file data as it gets uploaded. Upload
handlers are initially defined in the :setting:`FILE_UPLOAD_HANDLERS` setting,
which defaults to::
("django.core.files.uploadhandler.MemoryFileUploadHandler",
"django.core.files.uploadhandler.TemporaryFileUploadHandler",)
Together the ``MemoryFileUploadHandler`` and ``TemporaryFileUploadHandler``
provide Django's default file upload behavior of reading small files into memory
and large ones onto disk.
You can write custom handlers that customize how Django handles files. You
could, for example, use custom handlers to enforce user-level quotas, compress
data on the fly, render progress bars, and even send data to another storage
location directly without storing it locally.
.. _modifying_upload_handlers_on_the_fly:
process.
Modifying upload handlers on the fly
------------------------------------
@ -371,121 +274,3 @@ list::
@csrf_protect
def _upload_file_view(request):
... # Process request
Writing custom upload handlers
------------------------------
.. currentmodule:: django.core.files.uploadhandler
.. class:: FileUploadHandler
All file upload handlers should be subclasses of
``django.core.files.uploadhandler.FileUploadHandler``. You can define upload
handlers wherever you wish.
Required methods
~~~~~~~~~~~~~~~~
Custom file upload handlers **must** define the following methods:
.. method:: FileUploadHandler.receive_data_chunk(raw_data, start)
Receives a "chunk" of data from the file upload.
``raw_data`` is a byte string containing the uploaded data.
``start`` is the position in the file where this ``raw_data`` chunk
begins.
The data you return will get fed into the subsequent upload handlers'
``receive_data_chunk`` methods. In this way, one handler can be a
"filter" for other handlers.
Return ``None`` from ``receive_data_chunk`` to short-circuit remaining
upload handlers from getting this chunk. This is useful if you're
storing the uploaded data yourself and don't want future handlers to
store a copy of the data.
If you raise a ``StopUpload`` or a ``SkipFile`` exception, the upload
will abort or the file will be completely skipped.
.. method:: FileUploadHandler.file_complete(file_size)
Called when a file has finished uploading.
The handler should return an ``UploadedFile`` object that will be stored
in ``request.FILES``. Handlers may also return ``None`` to indicate that
the ``UploadedFile`` object should come from subsequent upload handlers.
Optional methods
~~~~~~~~~~~~~~~~
Custom upload handlers may also define any of the following optional methods or
attributes:
.. attribute:: FileUploadHandler.chunk_size
Size, in bytes, of the "chunks" Django should store into memory and feed
into the handler. That is, this attribute controls the size of chunks
fed into ``FileUploadHandler.receive_data_chunk``.
For maximum performance the chunk sizes should be divisible by ``4`` and
should not exceed 2 GB (2\ :sup:`31` bytes) in size. When there are
multiple chunk sizes provided by multiple handlers, Django will use the
smallest chunk size defined by any handler.
The default is 64*2\ :sup:`10` bytes, or 64 KB.
.. method:: FileUploadHandler.new_file(field_name, file_name, content_type, content_length, charset, content_type_extra)
Callback signaling that a new file upload is starting. This is called
before any data has been fed to any upload handlers.
``field_name`` is a string name of the file ``<input>`` field.
``file_name`` is the unicode filename that was provided by the browser.
``content_type`` is the MIME type provided by the browser -- E.g.
``'image/jpeg'``.
``content_length`` is the length of the image given by the browser.
Sometimes this won't be provided and will be ``None``.
``charset`` is the character set (i.e. ``utf8``) given by the browser.
Like ``content_length``, this sometimes won't be provided.
``content_type_extra`` is extra information about the file from the
``content-type`` header. See :attr:`UploadedFile.content_type_extra
<django.core.files.uploadedfile.UploadedFile.content_type_extra>`.
This method may raise a ``StopFutureHandlers`` exception to prevent
future handlers from handling this file.
.. versionadded:: 1.7
The ``content_type_extra`` parameter was added.
.. method:: FileUploadHandler.upload_complete()
Callback signaling that the entire upload (all files) has completed.
.. method:: FileUploadHandler.handle_raw_input(input_data, META, content_length, boundary, encoding)
Allows the handler to completely override the parsing of the raw
HTTP input.
``input_data`` is a file-like object that supports ``read()``-ing.
``META`` is the same object as ``request.META``.
``content_length`` is the length of the data in ``input_data``. Don't
read more than ``content_length`` bytes from ``input_data``.
``boundary`` is the MIME boundary for this request.
``encoding`` is the encoding of the request.
Return ``None`` if you want upload handling to continue, or a tuple of
``(POST, FILES)`` if you want to return the new data structures suitable
for the request directly.