Do not wait Connection Attempt Delay without in progress fds
Reset Connection Attempt Delay when connection fails and there is no other socket connection in progress.
This is intended to resolve an issue that was temporarily worked around in Pull Request #12062.
`TCPServer::new` (used in tests such as `TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`) can only connect over either IPv6 or IPv4, depending on the environment.
Since HEv2 attempts to connect over IPv6 first, environments where IPv6 connections are unavailable return ECONNREFUSED immediately.
In such cases, the client should immediately retry the connection over IPv4.
However, HEv2 includes a specification for a "Connection Attempt Delay," where it waits 250ms after the previous connection attempt before starting the next one.
This delay causes Net::OpenTimeout (100ms) to be exceeded while waiting for the next connection attempt to start.
With this change, when a connection attempt fails, if there are sockets still attempting to connect and there are addresses yet to be tried, the Connection Attempt Delay will be resetted, allowing the next connection attempt to start immediately.
---
Additionally, the following minor fixes have been made:
- The `nfds` value used for select(2) is now reset with each wait.
with a TCPSoerver that is only listening
to avoid AssertionFailedError on Ubuntu.
---
The tests such as
`TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`
expect to raise a `Net::WriteTimeout` due to a failure in writing to the server.
However, on Ubuntu environments,
the server immediately returns a "Connection Refused" in such cases.
The socket created with `TCPSocket.new` that supports HEv2 catches this immediately
and raises a `Net::OpenTimeout`.
As a result, these tests fail due to raising a different exception than expected.
This PR adds `Net::OpenTimeout` asexceptions to avoid these test failures.
If a socket error occurs while performing a streaming download via
the response block provided to transport_request, avoid calling
the response block again as this would result in duplicate data
received by the client.
Fixes https://github.com/ruby/net-http/pull/86
Fixes https://github.com/ruby/net-http/pull/87
Fixes [Bug #11526]
114d01b092
Co-authored-by: Jeremy Stanley <jeremy@instructure.com>
* no_proxy is meant to operate on the destination address, not on the name of the proxy
* if both end with `'.example'`, the test does not nail down the behaviour
bb9a5cfa3d
This list is out of date. At least OpenBSD since 2013 does not
allow one user to read the environment variables of a process
run by another user.
While we could try to keep the list updated, I think it's a bad
idea to not use the user/password from the environment, even if
another user on the system could read it. If http_proxy exists
in the environment, and other users can read it, it doesn't
make it more secure for Ruby to ignore it. You could argue that
it encourages poor security practices, but net/http should provide
mechanism, not policy.
Fixes [Bug #18908]
1e4585153d
The ignore_eof setting on HTTPResponse makes it so an EOFError is
raised when reading bodies with a defined Content-Length, if the
body read was truncated due to the socket be closed.
The ignore_eof setting on HTTP sets the values used in responses
that are created by the object.
For backwards compatibility, the default is for both settings is
true. However, unless you are specifically tested for and handling
truncated responses, it's a good idea to set ignore_eof to false so
that errors are raised for truncated responses, instead of those
errors silently being ignored.
Fixes [Bug #14972]
4d47e34995
This allows for the ability to opt-in to a method to set the
encoding of response bodies. By setting the accessor to a String
or Encoding instance, it will use the specified encoding.
Setting the value of true will try to detect the encoding of the
response body, either using the Content-Type header (assuming it
specifies charset) or by scanning for a <meta> tag in the document
that specifies the encoding. The default is false in which case
no forcing of encoding will be done (same as before the patch).
Implements [Feature #2567]
Implements [Feature #15517]
6233e6b7c1
Co-authored-by: Yui Naruse <naruse@ruby-lang.org>
The last_communicated timestamp is for HTTP persistent connection, to
decide whether the current TCP connection may be reused for the
subsequent requests or not. Naturally, the timer must be reset if the
connection is recreated since it is no longer relevant.
0a013de42d
The existing implementation of `Net::HTTP#write_timeout` relies on
`Net::BefferedIO` to trigger the `Net::WriteTimeout` error. This commit
changes `send_request_with_body_stream` to remove the optimization that
was making `Net::HTTP#write_timeout` not work when `body_stream` is
used.
Open issue:
https://bugs.ruby-lang.org/issues/17933a0fab1ab52
If someone sets an env variable defining a http_proxy, containing a
username / password with percent-encoded characters, then the resulting
base64 encoded auth header will be wrong.
For example, suppose a username is `Y\X` and the password is `R%S] ?X`.
Properly URL encoded the proxy url would be:
http://Y%5CX:R%25S%5D%20%3FX@proxy.example:8000
The resulting proxy auth header should be: `WVxYOlIlU10gP1g=`, but the
getters defined by ruby StdLib `URI` return a username `Y%5CX` and
password `R%25S%5D%20%3FX`, resulting in `WSU1Q1g6UiUyNVMlNUQlMjAlM0ZY`.
As a result the proxy will deny the request.
Please note that this is my first contribution to the ruby ecosystem, to
standard lib especially and I am not a ruby developer.
References:
- https://gitlab.com/gitlab-org/gitlab/-/issues/289836
- https://bugs.ruby-lang.org/projects/ruby-master/repository/trunk/revisions/58461
- https://bugs.ruby-lang.org/issues/17542e57d4f38aa
The test fails randomly on the CI of OpenCSW SPARC Solaris 10:
20200321T091909Z.fail.html.gz
```
1) Failure:
TestNetHTTP_v1_2_chunked#test_timeout_during_HTTP_session [/export/home/rubyci/unstable10s/tmp/build/20200321T091909Z/ruby/test/net/http/test_http.rb:575]:
[Net::ReadTimeout] exception expected, not #<Net::OpenTimeout: execution expired>.
```
The environment uses RUBY_TEST_SUBPROCESS_TIMEOUT_SCALE=20, so the open
timeout 0.1 sec. means 2.0 sec. for the environment, but it seems too
strict because the environment is painfully slow.
* Let Net::HTTP.get take request headers
* Add more test cases for no header usages
* Add examples with request headers
* Add a NEWS entry [ci skip]
[Feature #16686]
Because the test fails under HTTP proxy settings.
20191210T000004Z.fail.html.gz
```
1) Failure:
TestNetHTTPS#test_get_SNI_failure [/export/home/users/chkbuild/cb-gcc/tmp/build/20191210T000004Z/ruby/test/net/http/test_https.rb:81]:
[OpenSSL::SSL::SSLError] exception expected, not #<Net::HTTPServerException: 403 "Forbidden">.
```
The new SNI feature introduced at 54072e329c may need to be improved for
HTTP proxy environment.
An EPIPE when sending the request should be ignored. Even if you
cannot write more data, you may still be able to read the server's
response.
Fixes [Bug #14466]
This test is about write_timeout.
To ensure it really raised Net::WriteTimeout, skip this test on Windows,
whose write returns immediately even for large data.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/net/http/test_http.rb (test_timeout_during_HTTP_session_write): on some
platforms such as Windows immediately returns from Socket#write, and have to
wait to read its response. So, we can not handle Net::WriteTimeout and should
handle Net::ReadTimeout instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63597 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/net/http/test_http.rb (test_s_start): git rid of error when
failed to start a connection.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/net/http/test_http.rb (test_s_start): finish connections to
get rid of FD leaks.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Note that this feature is enabled only on environment variables are
multi-user safe. In this time the list includes Linux, FreeBSD, or
Darwin. [Bug #12921]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58461 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
in 1xx (HTTPInformation) also needs to continue. [Bug #12890]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56596 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
