The ignore_eof setting on HTTPResponse makes it so an EOFError is
raised when reading bodies with a defined Content-Length, if the
body read was truncated due to the socket be closed.
The ignore_eof setting on HTTP sets the values used in responses
that are created by the object.
For backwards compatibility, the default is for both settings is
true. However, unless you are specifically tested for and handling
truncated responses, it's a good idea to set ignore_eof to false so
that errors are raised for truncated responses, instead of those
errors silently being ignored.
Fixes [Bug #14972]
4d47e34995
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Don't delete content-length before yielding inflate body, as that
causes a switch to read the entire body instead of reading in
chunks.
Fixes [Bug #16672]
58284e9710
Co-authored-by: st0012 <stan001212@gmail.com>
This allows for the ability to opt-in to a method to set the
encoding of response bodies. By setting the accessor to a String
or Encoding instance, it will use the specified encoding.
Setting the value of true will try to detect the encoding of the
response body, either using the Content-Type header (assuming it
specifies charset) or by scanning for a <meta> tag in the document
that specifies the encoding. The default is false in which case
no forcing of encoding will be done (same as before the patch).
Implements [Feature #2567]
Implements [Feature #15517]
6233e6b7c1
Co-authored-by: Yui Naruse <naruse@ruby-lang.org>
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Fixes [Bug #16672]
a7cb30124c
This reverts commit 5105240b1e.
In RFC 2616:
```
deflate
The "zlib" format defined in RFC 1950 [31] in combination with
the "deflate" compression mechanism described in RFC 1951 [29].
```
So "Content-Encoding: deflate" means zlib format, not raw deflate.
[Bug #11268]
Net::HTTP had used `Zlib::Inflate.new(32 + Zlib::MAX_WBITS)` for all
content encoding (deflate, zlib, and gzip).
But the argument `32 + Zlib::MAX_WBITS` means zlib and gzip decoding
with automatic header detection, so (raw) deflate compression had not
been supported.
This change makes it support raw deflate correctly by passing an
argument `-Zlib::MAX_WBITS` (which means raw deflate) to
`Zlib::Inflate.new`. All deflate-mode tests are fixed too.
[Bug #11268]
* Make links from Net::GenericHTTPRequest work;
* Document +dest+ param of HTTPResponse#read_body;
* Slightly improve reference to particular response
classes from HTTPResponse class docs.
We no longer need the compressed data once the inflate block is
called; so clear it ASAP to reduce memory overhead. This is a
small chunk, so it only saves a few hundred kilobytes with the
script below.
before: RssAnon: 5976 kB
after: RssAnon: 5564 kB
------
require 'net/http'
require 'zlib'
response_gz = ARGV.shift or abort "#$0 TEMPORARY_FILE"
# pre-create response since compressing is slower than decompressing
unless File.readable?(response_gz)
nr = 16384 * 2
buf = ((0..255).map(&:chr).join * 128)
File.open(response_gz, 'wb') do |fp|
gzip = Zlib::GzipWriter.new(fp)
nr.times { gzip.write(buf) }
gzip.close
end
buf.clear
end
response_gz = File.open(response_gz)
s = TCPServer.new('127.0.0.1', 0)
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.write("HTTP/1.1 200 OK\r\n" \
"Content-Length: #{response_gz.stat.size}\r\n" \
"Content-Encoding: gzip\r\n" \
"Accept-Ranges: bytes\r\n" \
"\r\n")
IO.copy_stream(response_gz, c)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get(-'/') do |res|
res.read_body(&:clear)
end
end
puts File.readlines(-'/proc/self/status').grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/http/response.rb (inflate_adapter): clear compressed_chunk
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61684 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
If uplevel keyword is given, the warning message is prepended
with caller file and line information and the string "warning: ".
The use of the uplevel keyword makes Kernel#warn format output
similar to how rb_warn formats output.
This patch modifies net/ftp and net/imap to use Kernel#warn
instead of $stderr.puts or $stderr.printf, since they are used
for printing warnings.
This makes lib/cgi/core and tempfile use $stderr.puts instead of
warn for debug logging, since they are used for debug printing
and not for warning.
This does not modify bundler, rubygems, or rdoc, as those are
maintained outside of ruby and probably wish to remain backwards
compatible with older ruby versions.
rb_warn_m code is originally from nobu, but I've changed it
so that it only includes the path and lineno from uplevel
(not the method), and also prepends the string "warning: ",
to make it more similar to rb_warn.
From: Jeremy Evans code@jeremyevans.net
Signed-off-by: Urabe Shyouhei shyouhei@ruby-lang.org
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61155 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When you change this to true, you may need to add more tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
fix a bug that empty gzipped response body causes Zlib::BufError.
[ruby-core:68846] [Bug #11058]
* test/net/http/test_httpresponse.rb: tests for the above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51465 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
raise first exception even if inflate_body_io.finish raises error.
when begin block raises error, finish usually raises error too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46681 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Accept-Encoding. This allows users to handle Content-Encoding for
themselves. This restores backwards-compatibility with Ruby 1.x.
* lib/net/http/generic_request.rb: ditto.
* lib/net/http/response.rb: ditto
* test/net/http/test_http.rb: Test for the above.
* test/net/http/test_http_request.rb: ditto.
* test/net/http/test_httpresponse.rb: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
deflate-encoded response bodies. [Feature #6942]
* lib/net/http/generic_request.rb: Automatically accept gzip and
deflate content-encoding for requests. [Feature #6494]
* lib/net/http/request.rb: Updated documentation for #6494.
* lib/net/http.rb: Updated documentation for #6492 and #6494, removed
Content-Encoding handling now present in Net::HTTPResponse.
* test/net/http/test_httpresponse.rb: Tests for #6492
* test/net/http/test_http_request.rb: Tests for #6494
* test/open-uri/test_open-uri.rb (test_content_encoding): Updated test
for automatic content-encoding handling.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
pointer.
* lib/net/http/responses.rb: Add RFC numbers to base on.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36017 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
