If a socket error occurs while performing a streaming download via
the response block provided to transport_request, avoid calling
the response block again as this would result in duplicate data
received by the client.
Fixes https://github.com/ruby/net-http/pull/86
Fixes https://github.com/ruby/net-http/pull/87
Fixes [Bug #11526]
114d01b092
Co-authored-by: Jeremy Stanley <jeremy@instructure.com>
* no_proxy is meant to operate on the destination address, not on the name of the proxy
* if both end with `'.example'`, the test does not nail down the behaviour
bb9a5cfa3d
This comment previously specified TLS 1.2, but actually set the
version to TLS 1.0. LibreSSL 3.8.1 (included in OpenBSD 7.4)
dropped support for TLS 1.0/1.1 for security reasons, which
broke this test. Switch the test to use TLS 1.2 as documented
so it will continue to work on OpenBSD 7.4+.
97be4de53a
It's questionable whether we want to allow rstrip to work for strings
where the broken coderange occurs before the trailing whitespace and
not after, but this approach is probably simpler, and I don't think
users should expect string operations like rstrip to work on broken
strings.
In some cases, this changes rstrip to raise
Encoding::CompatibilityError instead of ArgumentError. However, as
the problem is related to an encoding issue in the receiver, and due
not due to an issue with an argument, I think
Encoding::CompatibilityError is the more appropriate error.
Fixes [Bug #18931]
This list is out of date. At least OpenBSD since 2013 does not
allow one user to read the environment variables of a process
run by another user.
While we could try to keep the list updated, I think it's a bad
idea to not use the user/password from the environment, even if
another user on the system could read it. If http_proxy exists
in the environment, and other users can read it, it doesn't
make it more secure for Ruby to ignore it. You could argue that
it encourages poor security practices, but net/http should provide
mechanism, not policy.
Fixes [Bug #18908]
1e4585153d
* Returning nil from the `content_range` method instead of raising an
error when the unit in the content-range header is not "bytes".
Fix https://bugs.ruby-lang.org/issues/114500b5030dd86
Co-Authored-By: Nobuyoshi Nakada <nobu@ruby-lang.org>
* Added a nil check in Net::HTTPHeader#initialize_http_header for keys in the header that do not have any value
* Returning nil from the content_range method instead of raising an error when the unit in the content-range header is not bytes
* Modified initialize_http_header to match trunk
fix [Bug #11450]
fix https://github.com/ruby/ruby/pull/1018
The ignore_eof setting on HTTPResponse makes it so an EOFError is
raised when reading bodies with a defined Content-Length, if the
body read was truncated due to the socket be closed.
The ignore_eof setting on HTTP sets the values used in responses
that are created by the object.
For backwards compatibility, the default is for both settings is
true. However, unless you are specifically tested for and handling
truncated responses, it's a good idea to set ignore_eof to false so
that errors are raised for truncated responses, instead of those
errors silently being ignored.
Fixes [Bug #14972]
4d47e34995
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Don't delete content-length before yielding inflate body, as that
causes a switch to read the entire body instead of reading in
chunks.
Fixes [Bug #16672]
58284e9710
Co-authored-by: st0012 <stan001212@gmail.com>
This allows for the ability to opt-in to a method to set the
encoding of response bodies. By setting the accessor to a String
or Encoding instance, it will use the specified encoding.
Setting the value of true will try to detect the encoding of the
response body, either using the Content-Type header (assuming it
specifies charset) or by scanning for a <meta> tag in the document
that specifies the encoding. The default is false in which case
no forcing of encoding will be done (same as before the patch).
Implements [Feature #2567]
Implements [Feature #15517]
6233e6b7c1
Co-authored-by: Yui Naruse <naruse@ruby-lang.org>
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Fixes [Bug #16672]
a7cb30124c
RFC 6066, section 3, explicitly disallows the use of an IP address
as an SNI server name. So check if the connection is being made
to an IP address using the resolv regexps, and do not set an SNI
hostname in that case.
Recent changes to LibreSSL make it more strictly follow RFC 6066,
resulting an s.hostname= raising an error if passed an IP address.
When such verions of LibreSSL are used, this change not only fixes
the net/http tests, it also fixes tests for webrick and open-uri,
which both make SSL connections to 127.0.0.1 using net/http in
their tests.
Avoid warning in the openssl extension by unsetting
@ssl_context.verify_hostname if connecting to an IP address.
Make changes so that the post_connection_check still happens
when connecting to an IP address, which is necessary to keep
checking that the certificate returned includes the IP address,
which one of the tests depends on.
Revert the previous change that modified the regexp used for
checking the error message.
fa68e64bee
Server Name Indication does not allow IP addresses (RFC 6066, section 3:
`Literal IPv4 and IPv6 addresses are not permitted in "HostName".`).
Recent versions of LibreSSL enforce this restriction, which raises
when setting the hostname to an IP address (s.hostname=), before
attempting to setup the SSL connection.
Since the certificate used in the test is already for localhost,
we cannot connect to localhost. However, as all the test does is
check for a specific exception, just modify the regexp used so it
handles both cases.
6ab399db45
52ab9bbee9 makes a failure.
```
1) Failure:
HTTPRequestTest#test_initialize_HEAD [/tmp/ruby/v3/src/trunk/test/net/http/test_http_request.rb:53]:
<{"accept"=>["*/*"], "user-agent"=>["Ruby"]}> expected but was
<{"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"],
"accept"=>["*/*"],
"user-agent"=>["Ruby"]}>.
```
I'm not sure it is correct, so please review later.
The last_communicated timestamp is for HTTP persistent connection, to
decide whether the current TCP connection may be reused for the
subsequent requests or not. Naturally, the timer must be reset if the
connection is recreated since it is no longer relevant.
0a013de42d
The existing implementation of `Net::HTTP#write_timeout` relies on
`Net::BefferedIO` to trigger the `Net::WriteTimeout` error. This commit
changes `send_request_with_body_stream` to remove the optimization that
was making `Net::HTTP#write_timeout` not work when `body_stream` is
used.
Open issue:
https://bugs.ruby-lang.org/issues/17933a0fab1ab52
It is not used in net/http library code since commit 15ccd0118c13
(r36473 in ruby svn trunk, 2012).
require's in test suite are also cleaned up.
996d18a43f
If someone sets an env variable defining a http_proxy, containing a
username / password with percent-encoded characters, then the resulting
base64 encoded auth header will be wrong.
For example, suppose a username is `Y\X` and the password is `R%S] ?X`.
Properly URL encoded the proxy url would be:
http://Y%5CX:R%25S%5D%20%3FX@proxy.example:8000
The resulting proxy auth header should be: `WVxYOlIlU10gP1g=`, but the
getters defined by ruby StdLib `URI` return a username `Y%5CX` and
password `R%25S%5D%20%3FX`, resulting in `WSU1Q1g6UiUyNVMlNUQlMjAlM0ZY`.
As a result the proxy will deny the request.
Please note that this is my first contribution to the ruby ecosystem, to
standard lib especially and I am not a ruby developer.
References:
- https://gitlab.com/gitlab-org/gitlab/-/issues/289836
- https://bugs.ruby-lang.org/projects/ruby-master/repository/trunk/revisions/58461
- https://bugs.ruby-lang.org/issues/17542e57d4f38aa
because the name "MJIT" is an internal code name, it's inconsistent with
--jit while they are related to each other, and I want to discourage future
JIT implementation-specific (e.g. MJIT-specific) APIs by this rename.
[Feature #17490]
On some environments that uses OpenSSL 1.1.1h, the two tests now fail.
20200924T062352Z.fail.html.gz
1159288773
```
1) Failure:
TestNetHTTPS#test_get [/data/data/com.termux/files/home/cb/tmp/build/20200924T062352Z/ruby/test/net/http/test_https.rb:47]:
<"0\x82\x03\xED0\x82\x02\xD5\xA0\x03..."> expected but was
<"0\x82\x03\xE30\x82\x02\xCB\xA0\x03...">.
```
Not sure why, but verify_callback now seems to receive only SERVER_CERT
but not CA_CERT.
It would be good to investigate the issue furthermore, but tentatively,
I want to stop the failures.
invalid_servername is not a valid name in an SSL request due to
the use of the underscore, and LibreSSL 3.2.0 will raise an
exception for this. These tests are not testing the allowed
characters in the server name, but how net/http handles cases where
the server name provided does not match the IP address you are
trying to connect to, so I think it's better to just modify the
tests to use a correct format.
While here, fix a typo in a test name, and use better code in the
ensure block so the same test doesn't issue both a failure and an
error.
0e8dc91120
The test fails randomly on the CI of OpenCSW SPARC Solaris 10:
20200321T091909Z.fail.html.gz
```
1) Failure:
TestNetHTTP_v1_2_chunked#test_timeout_during_HTTP_session [/export/home/rubyci/unstable10s/tmp/build/20200321T091909Z/ruby/test/net/http/test_http.rb:575]:
[Net::ReadTimeout] exception expected, not #<Net::OpenTimeout: execution expired>.
```
The environment uses RUBY_TEST_SUBPROCESS_TIMEOUT_SCALE=20, so the open
timeout 0.1 sec. means 2.0 sec. for the environment, but it seems too
strict because the environment is painfully slow.
