Because hard to specify commits related to r67479 only.
So please commit again.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit should fix Net::Protocol::BufferedIO#write when sending
large multi-byte string like following example.
```
$ ruby -rnet/http -rjson -v -e "Net::HTTP.post(URI('http://httpbin.org/post'), { text: 'あ'*100_000 }.to_json, 'Content-Type' => 'application/json')"
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Traceback (most recent call last):
19: from -e:1:in `<main>'
18: from lib/ruby/2.6.0/net/http.rb:500:in `post'
17: from lib/ruby/2.6.0/net/http.rb:605:in `start'
16: from lib/ruby/2.6.0/net/http.rb:920:in `start'
15: from lib/ruby/2.6.0/net/http.rb:502:in `block in post'
14: from lib/ruby/2.6.0/net/http.rb:1281:in `post'
13: from lib/ruby/2.6.0/net/http.rb:1493:in `send_entity'
12: from lib/ruby/2.6.0/net/http.rb:1479:in `request'
11: from lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
10: from lib/ruby/2.6.0/net/http.rb:1506:in `catch'
9: from lib/ruby/2.6.0/net/http.rb:1507:in `block in transport_request'
8: from lib/ruby/2.6.0/net/http/generic_request.rb:123:in `exec'
7: from lib/ruby/2.6.0/net/http/generic_request.rb:189:in `send_request_with_body'
6: from lib/ruby/2.6.0/net/protocol.rb:247:in `write'
5: from lib/ruby/2.6.0/net/protocol.rb:265:in `writing'
4: from lib/ruby/2.6.0/net/protocol.rb:248:in `block in write'
3: from lib/ruby/2.6.0/net/protocol.rb:275:in `write0'
2: from lib/ruby/2.6.0/net/protocol.rb:275:in `each_with_index'
1: from lib/ruby/2.6.0/net/protocol.rb:275:in `each'
lib/ruby/2.6.0/net/protocol.rb:280:in `block in write0': undefined method `bytesize' for nil:NilClass (NoMethodError)
```
[Fix GH-2058]
From: Eito Katagiri <eitoball@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
So that classes which uses Net::HTTP with https can use OpenSSL
namespace for example exception classes like OpenSSL::SSL::SSLError.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66401 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Stringify after downcase, to restrict arguments.
[Bug#15394]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66321 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/net/protocol.rb (ReadTimeout, WriteTimeout): Net::ReadTimeout and Net::WriteTimeout should tell the cause socket
[Feature #14832] [ruby-core:87440]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/net/http.rb: Documentation for Net::HTTP claims that
multipart/form-data is not supported, but Net::HTTPHeader#set_form
supports it since 1.9.3.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65230 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When TLS 1.3 is in use, the session ticket may not have been sent yet
even though a handshake has finished. Also, the ticket could change if
multiple session ticket messages are sent by the server. Use
SSLContext#session_new_cb instead of calling SSLSocket#session
immediately after a handshake. This way also works with earlier protocol
versions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64234 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/net/http/exceptions.rb: revert a part of r63590. to deprecate
Net::ProtoServerError seems to be wrong.
see [ruby-core:87488] [Feature#14688]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63661 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`Net::HTTPServerException` has been deprecated since r63590. [Bug #14688]
And `net/http/responses.rb` uses the deprecated constant, so Ruby warns of the deprecation.
Example:
```bash
$ ruby -r net/http -e ''
/home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/net/http/responses.rb:22: warning: constant Net::HTTPServerException is deprecated
```
This change suppresses the warning.
From: Masataka Pocke Kuwabara <kuwabara@pocke.me>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63591 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to ShockwaveNN (Pavel Lobashov) for reporting the bug.
[ruby-core:86990] [Bug #14750]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Otherwise connections (commonly on IDLE, but it could be any
command) may never receive notifications of link errors.
[ruby-core:86628] [Feature #14703]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Paul Kuruvilla <rohitpaulk@gmail.com> for the patch
* lib/net/http.rb: fix documentation for HTTP connection reuse
[ruby-core:84815] [Bug #14349]
From: Paul Kuruvilla <rohitpaulk@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Michael Zimmerman for the bug report
* lib/net/pop.rb: make modified strings mutable
[ruby-core:85210] [Bug #14416]
* test/net/pop/test_pop.rb: new test
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reduces both user and system CPU time for large
uploads with dynamically-generated request bodies.
user system total real
before: 0.393334 1.580000 1.973334 ( 1.971066)
after: 0.223334 0.976666 1.200000 ( 1.198514)
------
require 'socket'
require 'net/http'
require 'benchmark'
nr = 1024 * 1024 * 1024
s = TCPServer.new('127.0.0.1', 0)
addr = s.addr
at_exit { Process.waitall }
fork do
c = s.accept
# not exactly accurate but fast
IO.copy_stream(c, '/dev/null', nr + 500000)
begin
buf = c.readpartial(16384)
tmp = ''
until buf.end_with?(-"0\r\n\r\n")
buf << c.readpartial(16384, tmp)
end
rescue EOFError
end
c.write "HTTP/1.1 201 Created\r\nConnection:close\r\n\r\n"
c.close
end
r, w = IO.pipe
fork do
r.close
IO.copy_stream('/dev/zero', w, nr)
w.close
end
w.close
Net::HTTP.start(addr[3], addr[1]) do |http|
put = Net::HTTP::Put.new('/dev0/foo')
put['Content-Type'] = 'application/content-type'
put['Transfer-Encoding'] = 'chunked'
put.body_stream = r
puts(Benchmark.measure { http.request(put) })
end
------
* lib/net/http/generic_request.rb (write): use multi-arg write
* lib/net/protocol.rb (write): support multi-arg
(write0): ditto
[ruby-core:84845] [Feature #14339]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61812 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We no longer need the compressed data once the inflate block is
called; so clear it ASAP to reduce memory overhead. This is a
small chunk, so it only saves a few hundred kilobytes with the
script below.
before: RssAnon: 5976 kB
after: RssAnon: 5564 kB
------
require 'net/http'
require 'zlib'
response_gz = ARGV.shift or abort "#$0 TEMPORARY_FILE"
# pre-create response since compressing is slower than decompressing
unless File.readable?(response_gz)
nr = 16384 * 2
buf = ((0..255).map(&:chr).join * 128)
File.open(response_gz, 'wb') do |fp|
gzip = Zlib::GzipWriter.new(fp)
nr.times { gzip.write(buf) }
gzip.close
end
buf.clear
end
response_gz = File.open(response_gz)
s = TCPServer.new('127.0.0.1', 0)
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.write("HTTP/1.1 200 OK\r\n" \
"Content-Length: #{response_gz.stat.size}\r\n" \
"Content-Encoding: gzip\r\n" \
"Accept-Ranges: bytes\r\n" \
"\r\n")
IO.copy_stream(response_gz, c)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get(-'/') do |res|
res.read_body(&:clear)
end
end
puts File.readlines(-'/proc/self/status').grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/http/response.rb (inflate_adapter): clear compressed_chunk
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61684 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's no need to allocate a temporary string when @rbuf is
empty, we can use it as the read_nonblock destination buffer to
save both allocation overhead and avoid a later memcpy.
This results in a halving user CPU time and tiny memory
reduction with the script below:
user system total real
before 0.603333 0.539999 1.143332 ( 1.143347)
RssAnon: 5624 kB
after 0.283334 0.560000 0.843334 ( 0.846072)
RssAnon: 5592 kB
------
require 'net/http'
require 'benchmark'
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024 * 2
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.send("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n", Socket::MSG_MORE)
IO.copy_stream('/dev/zero', c, len)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
puts(Benchmark.measure { res.read_body(&:clear) })
end
end
puts File.readlines("/proc/self/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/protocol.rb (rbuf_fill): avoid allocation if rbuf is empty
[ruby-core:84678] [Feature #14326]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not an encoding expert, but this seems necessary for the next
change. All of the IO#read, IO#read_nonblock and related
methods will return a binary string when given a length
argument; so anything appended to these buffers via <<
will be binary.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Oops, not ready, yet (will work on this tomorrow :x).
This reverts commit r61638
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I noticed this bug while working on something else with
RUBYOPT=-d on, existing test cases all passed with it.
Note: I use String.new because it is the local style, here,
I prefer +'' (or ''.b, for a future commit)
* lib/net/ftp.rb (BufferedSocket#read): use String.new
* test/net/ftp/test_buffered_socket.rb (test_read_nil): new test
[ruby-core:84675] [Bug #14323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not an encoding expert, but this seems necessary for the next
change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.
The following script exhibits reduced memory usage and
runtimes using the time(1) command:
2.9s => 2.6s
70MB => 12 MB
---------
require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end
end
----------
* lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len
[Feature #14268]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61602 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
like r59693, initialize_http_header also should raise error. [Bug #14208]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
