make_bytes() assumed that if the Content-Encoding header is set, then
everything had already been dealt with bytes-wise, but in a streaming
situation this was not necessarily the case.
make_bytes() is only called when necessary when working with a
StreamingHttpResponse iterable, but by that point the middleware has
added the Content-Encoding header and thus make_bytes() tried to call
bytes(value) (and dies). If it had been a normal HttpResponse,
make_bytes() would have been called when the content was set, well
before the middleware set the Content-Encoding header.
This commit removes the special casing when Content-Encoding is set,
allowing unicode strings to be encoded during the iteration before they
are e.g. gzipped. This behaviour was added a long time ago for #4969 and
it doesn't appear to be necessary any more, as everything is correctly
made into bytes at the appropriate places.
Two new tests, to show that supplying non-ASCII characters to a
StreamingHttpResponse works fine normally, and when passed through the
GZip middleware (the latter dies without the change to make_bytes()).
Removes the test with a nonsense Content-Encoding and Unicode input - if
this were to happen, it can still be encoded as bytes fine.
Added getvalue() to HttpResponse to return the content of the response,
along with a few other methods to partially match io.IOBase.
Thanks Claude Paroz for the suggestion and Nick Sanford for review.
Fixed#20187 -- Allowed repeated iteration of HttpResponse.
All this became possible when support for old-style streaming responses was
finally removed.
Passed large maxlinelen to email.Header to prevent newlines from being
inserted into value returned by _convert_to_charset
Thanks mjl at laubach.at for the report.
Tim Graham
Matthew Somerville
Michael Kelly
Berker Peksag
Claude Paroz
Duncan Parkes
Aymeric Augustin
Lukasz Balcerzak
Jason Myers
coagulant
e0ne
Daniel Boeve
Baptiste Mispelon
Florian Apolloner