The shutdown process here attempted to terminate the test server
by interrupting it with Thread#kill, and then proceeded to close
the server and join the thread. The kill does indeed interrupt
the accept call, but the close call could also interrupt the
thread as part of notifying blocked threads waiting on that
socket call.
In JRuby, where all of this can happen at the same time, it leads
to the following scenario:
* The server thread enters TCPServer#accept and blocks.
* The main thread calls Thread#kill to interrupt the accept call.
* The server thread wakes up and starts to propagate the kill.
There is a slight delay between this wakeup and removing the
server thread from the TCPServer's blocked threads list.
* The main thread calls TCPServer#close, which sees that the server
thread is still in the blocked list, so it initiates a second
interrupt to raise IOError "closed in another thread" on the
server thread.
* As the kill is bubbling out, another check for interrupts occurs,
causing it to see the new raise interrupt and propagate that
instead of the active kill.
* Because the server is now closed and the rescue here is empty,
the server loop will endlessly attempt and fail to call accept.
I was unable to determine how CRuby avoids this race. There may be
code that prevents an active kill interrupt from triggering
further interrupts.
In order to get these tests running on JRuby, I've made the
following changes:
* Only kill the thread; one interrupt is sufficient to break it
out of the accept call.
* Ensure outside the server loop that the server gets closed. This
happens within the server thread, so triggers no new interrupts.
* Minor cleanup for the pattern of using @ssl_server or @server.
This change avoids the race in JRuby (and possibly other parallel-
threaded implementations) and does not impact the behavior of the
tests.
54025b3870
Do not wait Connection Attempt Delay without in progress fds
Reset Connection Attempt Delay when connection fails and there is no other socket connection in progress.
This is intended to resolve an issue that was temporarily worked around in Pull Request #12062.
`TCPServer::new` (used in tests such as `TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`) can only connect over either IPv6 or IPv4, depending on the environment.
Since HEv2 attempts to connect over IPv6 first, environments where IPv6 connections are unavailable return ECONNREFUSED immediately.
In such cases, the client should immediately retry the connection over IPv4.
However, HEv2 includes a specification for a "Connection Attempt Delay," where it waits 250ms after the previous connection attempt before starting the next one.
This delay causes Net::OpenTimeout (100ms) to be exceeded while waiting for the next connection attempt to start.
With this change, when a connection attempt fails, if there are sockets still attempting to connect and there are addresses yet to be tried, the Connection Attempt Delay will be resetted, allowing the next connection attempt to start immediately.
---
Additionally, the following minor fixes have been made:
- The `nfds` value used for select(2) is now reset with each wait.
with a TCPSoerver that is only listening
to avoid AssertionFailedError on Ubuntu.
---
The tests such as
`TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`
expect to raise a `Net::WriteTimeout` due to a failure in writing to the server.
However, on Ubuntu environments,
the server immediately returns a "Connection Refused" in such cases.
The socket created with `TCPSocket.new` that supports HEv2 catches this immediately
and raises a `Net::OpenTimeout`.
As a result, these tests fail due to raising a different exception than expected.
This PR adds `Net::OpenTimeout` asexceptions to avoid these test failures.
If a socket error occurs while performing a streaming download via
the response block provided to transport_request, avoid calling
the response block again as this would result in duplicate data
received by the client.
Fixes https://github.com/ruby/net-http/pull/86
Fixes https://github.com/ruby/net-http/pull/87
Fixes [Bug #11526]
114d01b092
Co-authored-by: Jeremy Stanley <jeremy@instructure.com>
* no_proxy is meant to operate on the destination address, not on the name of the proxy
* if both end with `'.example'`, the test does not nail down the behaviour
bb9a5cfa3d
This comment previously specified TLS 1.2, but actually set the
version to TLS 1.0. LibreSSL 3.8.1 (included in OpenBSD 7.4)
dropped support for TLS 1.0/1.1 for security reasons, which
broke this test. Switch the test to use TLS 1.2 as documented
so it will continue to work on OpenBSD 7.4+.
97be4de53a
It's questionable whether we want to allow rstrip to work for strings
where the broken coderange occurs before the trailing whitespace and
not after, but this approach is probably simpler, and I don't think
users should expect string operations like rstrip to work on broken
strings.
In some cases, this changes rstrip to raise
Encoding::CompatibilityError instead of ArgumentError. However, as
the problem is related to an encoding issue in the receiver, and due
not due to an issue with an argument, I think
Encoding::CompatibilityError is the more appropriate error.
Fixes [Bug #18931]
This list is out of date. At least OpenBSD since 2013 does not
allow one user to read the environment variables of a process
run by another user.
While we could try to keep the list updated, I think it's a bad
idea to not use the user/password from the environment, even if
another user on the system could read it. If http_proxy exists
in the environment, and other users can read it, it doesn't
make it more secure for Ruby to ignore it. You could argue that
it encourages poor security practices, but net/http should provide
mechanism, not policy.
Fixes [Bug #18908]
1e4585153d
* Returning nil from the `content_range` method instead of raising an
error when the unit in the content-range header is not "bytes".
Fix https://bugs.ruby-lang.org/issues/114500b5030dd86
Co-Authored-By: Nobuyoshi Nakada <nobu@ruby-lang.org>
* Added a nil check in Net::HTTPHeader#initialize_http_header for keys in the header that do not have any value
* Returning nil from the content_range method instead of raising an error when the unit in the content-range header is not bytes
* Modified initialize_http_header to match trunk
fix [Bug #11450]
fix https://github.com/ruby/ruby/pull/1018
The ignore_eof setting on HTTPResponse makes it so an EOFError is
raised when reading bodies with a defined Content-Length, if the
body read was truncated due to the socket be closed.
The ignore_eof setting on HTTP sets the values used in responses
that are created by the object.
For backwards compatibility, the default is for both settings is
true. However, unless you are specifically tested for and handling
truncated responses, it's a good idea to set ignore_eof to false so
that errors are raised for truncated responses, instead of those
errors silently being ignored.
Fixes [Bug #14972]
4d47e34995
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Don't delete content-length before yielding inflate body, as that
causes a switch to read the entire body instead of reading in
chunks.
Fixes [Bug #16672]
58284e9710
Co-authored-by: st0012 <stan001212@gmail.com>
This allows for the ability to opt-in to a method to set the
encoding of response bodies. By setting the accessor to a String
or Encoding instance, it will use the specified encoding.
Setting the value of true will try to detect the encoding of the
response body, either using the Content-Type header (assuming it
specifies charset) or by scanning for a <meta> tag in the document
that specifies the encoding. The default is false in which case
no forcing of encoding will be done (same as before the patch).
Implements [Feature #2567]
Implements [Feature #15517]
6233e6b7c1
Co-authored-by: Yui Naruse <naruse@ruby-lang.org>
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Fixes [Bug #16672]
a7cb30124c
RFC 6066, section 3, explicitly disallows the use of an IP address
as an SNI server name. So check if the connection is being made
to an IP address using the resolv regexps, and do not set an SNI
hostname in that case.
Recent changes to LibreSSL make it more strictly follow RFC 6066,
resulting an s.hostname= raising an error if passed an IP address.
When such verions of LibreSSL are used, this change not only fixes
the net/http tests, it also fixes tests for webrick and open-uri,
which both make SSL connections to 127.0.0.1 using net/http in
their tests.
Avoid warning in the openssl extension by unsetting
@ssl_context.verify_hostname if connecting to an IP address.
Make changes so that the post_connection_check still happens
when connecting to an IP address, which is necessary to keep
checking that the certificate returned includes the IP address,
which one of the tests depends on.
Revert the previous change that modified the regexp used for
checking the error message.
fa68e64bee
