infect taint flag on Array#pack and String#unpack
with the directives "B", "b", "H" and "h".
* pack.c (pack_pack, pack_unpack_internal): infect taint flag.
* test/ruby/test_pack.rb: add test for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@65130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Support ubasecrt.dll 10.0.17763.1 included in Windows 10 October 2018 Update
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@65119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
webrick: support Proc objects as body responses
* lib/webrick/httpresponse.rb (send_body): call send_body_proc
(send_body_proc): new method
(class ChunkedWrapper): new class
* test/webrick/test_httpresponse.rb (test_send_body_proc): new test
(test_send_body_proc_chunked): ditto
[Feature #855]
webrick: favor .write over << method
This will make the next change to use IO.copy_stream
easier-to-read. When we can drop Ruby 2.4 support in a few
years, this will allow us to use writev(2) with multiple
arguments for headers and chunked responses.
* lib/webrick/cgi.rb (write): new wrapper method
lib/webrick/httpresponse.rb: (send_header): use socket.write
(send_body_io): ditto
(send_body_string): ditto
(send_body_proc): ditto
(_write_data): ditto
(ChunkedWrapper#write): ditto
(_send_file): ditto
------------------------------------------------------------------------
r62954 | normal | 2018-03-28 17:05:52 +0900 (水, 28 3 2018) | 14 lines
webrick/httpresponse: IO.copy_stream for regular files
Remove the redundant _send_file method since its functionality
is unnecessary with IO.copy_stream. IO.copy_stream also allows
the use of sendfile under some OSes to speed up copies to
non-TLS sockets.
Testing with "curl >/dev/null" and "ruby -run -e httpd" to
read a 1G file over Linux loopback reveals a reduction from
around ~0.770 to ~0.490 seconds on the client side.
* lib/webrick/httpresponse.rb (send_body_io): use IO.copy_stream
(_send_file): remove
[Feature #14237]
------------------------------------------------------------------------
r62955 | normal | 2018-03-28 17:05:57 +0900 (水, 28 3 2018) | 10 lines
webrick: use IO.copy_stream for single range response
This is also compatible with range responses generated
by Rack::File (tested with rack 2.0.3).
* lib/webrick/httpresponse.rb (send_body_io): use Content-Range
* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
use File object for the single range case
* test/webrick/test_filehandler.rb (get_res_body): use send_body
to test result
------------------------------------------------------------------------
r62956 | normal | 2018-03-28 17:06:02 +0900 (水, 28 3 2018) | 7 lines
test/webrick/test_filehandler.rb: stricter multipart range test
We need to ensure we generate compatibile output in
the face of future changes
* test/webrick/test_filehandler.rb (test_make_partial_content):
check response body
------------------------------------------------------------------------
r62957 | normal | 2018-03-28 17:06:08 +0900 (水, 28 3 2018) | 8 lines
webrick: quiet warning for multi-part ranges
Content-Length is ignored by WEBrick::HTTPResponse even if we
calculate it, so instead we chunk responses to HTTP/1.1 clients
and terminate HTTP/1.0 connections.
* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
quiet warning
------------------------------------------------------------------------
r62958 | normal | 2018-03-28 17:06:13 +0900 (水, 28 3 2018) | 7 lines
webrick/httpresponse: make ChunkedWrapper copy_stream-compatible
The .write method needs to return the number of bytes written
to avoid confusing IO.copy_stream.
* lib/webrick/httpresponse.rb (ChunkedWrapper#write): return bytes written
(ChunkedWrapper#<<): return self
------------------------------------------------------------------------
r62959 | normal | 2018-03-28 17:06:18 +0900 (水, 28 3 2018) | 9 lines
webrick: use IO.copy_stream for multipart response
Use the new Proc response body feature to generate a multipart
range response dynamically. We use a flat array to minimize
object overhead as much as possible; as many ranges may fit
into an HTTP request header.
* lib/webrick/httpservlet/filehandler.rb (multipart_body): new method
(make_partial_content): use multipart_body
get rid of test error/failure on Windows introduced at r62955
* lib/webrick/httpresponse.rb (send_body_io): use seek if NotImplementedError
is raised in IO.copy_stream with offset.
* lib/webrick/httpservlet/filehandler.rb (multipart_body): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@63014 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
dir.c: check NUL bytes
* dir.c (GlobPathValue): should be used in rb_push_glob only.
other methods should use FilePathValue.
https://hackerone.com/reports/302338
* dir.c (rb_push_glob): expand GlobPathValue
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62994 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
webrick: prevent response splitting and header injection
Original patch by tenderlove (with minor style adjustments).
* lib/webrick/httpresponse.rb (send_header): call check_header
(check_header): raise on embedded CRLF in header value
* test/webrick/test_httpresponse.rb
(test_prevent_response_splitting_headers): new test
* (test_prevent_response_splitting_cookie_headers): ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
webrick: use IO.copy_stream for multipart response
Use the new Proc response body feature to generate a multipart
range response dynamically. We use a flat array to minimize
object overhead as much as possible; as many ranges may fit
into an HTTP request header.
* lib/webrick/httpservlet/filehandler.rb (multipart_body): new method
(make_partial_content): use multipart_body
------------------------------------------------------------------------
r62960 | normal | 2018-03-28 17:06:23 +0900 (水, 28 3 2018) | 13 lines
webrick/httprequest: limit request headers size
We use the same 112 KB limit started (AFAIK) by Mongrel, Thin,
and Puma to prevent malicious users from using up all the memory
with a single request. This also limits the damage done by
excessive ranges in multipart Range: requests.
Due to the way we rely on IO#gets and the desire to keep
the code simple, the actual maximum header may be 4093 bytes
larger than 112 KB, but we're splitting hairs at that point.
* lib/webrick/httprequest.rb: define MAX_HEADER_LENGTH
(read_header): raise when headers exceed max length
------------------------------------------------------------------------
r62961 | normal | 2018-03-28 17:06:28 +0900 (水, 28 3 2018) | 9 lines
webrick/httpservlet/cgihandler: reduce memory use
WEBrick::HTTPRequest#body can be passed a block to process the
body in chunks. Use this feature to avoid building a giant
string in memory.
* lib/webrick/httpservlet/cgihandler.rb (do_GET):
avoid reading entire request body into memory
(do_POST is aliased to do_GET, so it handles bodies)
------------------------------------------------------------------------
r62962 | normal | 2018-03-28 17:06:34 +0900 (水, 28 3 2018) | 7 lines
webrick/httprequest: raise correct exception
"BadRequest" alone does not resolve correctly, it is in the
HTTPStatus namespace.
* lib/webrick/httprequest.rb (read_chunked): use correct exception
* test/webrick/test_httpserver.rb (test_eof_in_chunk): new test
------------------------------------------------------------------------
r62963 | normal | 2018-03-28 17:06:39 +0900 (水, 28 3 2018) | 9 lines
webrick/httprequest: use InputBufferSize for chunked requests
While WEBrick::HTTPRequest#body provides a Proc interface
for streaming large request bodies, clients must not force
the server to use an excessively large chunk size.
* lib/webrick/httprequest.rb (read_chunk_size): limit each
read and block.call to :InputBufferSize in config.
* test/webrick/test_httpserver.rb (test_big_chunks): new test
------------------------------------------------------------------------
r62964 | normal | 2018-03-28 17:06:44 +0900 (水, 28 3 2018) | 9 lines
webrick: add test for Digest auth-int
No changes to the actual code, this is a new test for
a feature for which no tests existed. I don't understand
the Digest authentication code well at all, but this is
necessary for the subsequent change.
* test/webrick/test_httpauth.rb (test_digest_auth_int): new test
(credentials_for_request): support bodies with POST
------------------------------------------------------------------------
r62965 | normal | 2018-03-28 17:06:49 +0900 (水, 28 3 2018) | 18 lines
webrick/httpauth/digestauth: stream req.body
WARNING! WARNING! WARNING! LIKELY BROKEN CHANGE
Pass a proc to WEBrick::HTTPRequest#body to avoid reading a
potentially large request body into memory during
authentication.
WARNING! this will break apps completely which want to do
something with the body besides calculating the MD5 digest
of it.
Also, keep in mind that probably nobody uses "auth-int".
Servers such as Apache, lighttpd, nginx don't seem to
support it; nor does curl when using POST/PUT bodies;
and we didn't have tests for it until now...
* lib/webrick/httpauth/digestauth.rb (_authenticate): stream req.body
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62970 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix setting method visibility on method wrapped with prepend
Ignore prepended modules when looking for already defined methods on a
class to set the visibility on.
[Fix GH-1834]
From: Dylan Thacker-Smith <Dylan.Smith@shopify.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62950 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
resolv.rb: close socket
* lib/resolv.rb (UnconnectedUDP#lazy_initialize): store new
sockets before binding, so the sockets get closed when the
requester is closing.
* lib/resolv.rb (ConnectedUDP#lazy_initialize): ditto.
* lib/resolv.rb (UnconnectedUDP#close): synchronize to get rid of
race condition.
* lib/resolv.rb (ConnectedUDP#close): ditto.
[ruby-core:85901] [Bug #14571]
From: quixoten (Devin Christensen) <quixoten@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62949 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
string.c: clear substring code range
* string.c (str_substr): substring of broken code range string may
be valid or broken. patch by tommy (Masahiro Tomita) at
[ruby-dev:50430] [Bug #14388].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62946 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
check array for zsuper. [Bug #14279]
* compile.c (iseq_compile_each0): for zsuper (NODE_ZSUPER), we need to check
given argument is Array or not.
* test/ruby/test_super.rb: add a test for this bug.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62944 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
net/ftp: fix FrozenError in BufferedSocket
I noticed this bug while working on something else with
RUBYOPT=-d on, existing test cases all passed with it.
Note: I use String.new because it is the local style, here,
I prefer +'' (or ''.b, for a future commit)
* lib/net/ftp.rb (BufferedSocket#read): use String.new
* test/net/ftp/test_buffered_socket.rb (test_read_nil): new test
[Bug #14323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62943 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
raise error if value contains CR/LF in iniheader of initialize_http_header
like r59693, initialize_http_header also should raise error. [Bug #14208]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62939 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
parse.y: end of script at newline
* parse.y (parser_yylex): deal with end of script chars just after
ignored newline as other places. [ruby-core:84349] [Bug #14206]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62938 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Removed the obstacle to running test-all with VC14.
* test/fiddle/test_import.rb (LIBC.fprintf): VC14's `fprintf` is very distinct
name, then, just ignore this test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62886 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
patched by Kazuki Yamaguchi <k@rhe.jp>
bio: prevent possible GC issue in ossl_obj2bio()
Prevent the new object created by StringValue() from being GCed.
Luckily, as none of the callers of ossl_obj2bio() reads from the
returned BIO after possible triggering GC, this has not been a real
problem.
As a bonus, ossl_protect_obj2bio() function which is no longer used
anywhere is removed.
merge f842b0d5c5
patched by Kazuki Yamaguchi <k@rhe.jp>
bio: do not use the FILE BIO method in ossl_obj2bio()
Read everything from an IO object into a String first and use the
memory buffer BIO method just as we do for String inputs.
For MSVC builds, the FILE BIO method uses the "UPLINK" interface that
requires the application to provide OPENSSL_Applink() function. For us,
the "application" means ruby.exe, in which we can't do anything. As a
workaround, avoid using the FILE BIO method at all.
Usually private keys or X.509 certificates aren't that large and the
temporarily increased memory usage hopefully won't be an issue.
ext/openssl/ossl_version.h (OpenSSL::VERSION): bump to 1.1.1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62885 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Support MSVC14 and 15 [Bug #11118]
Search _pioinfo which is not exported after MSVC14.
[Bug #12014] [GH-884]
win32.c: suppress warnings
* win32/win32.c (set_pioinfo_extra): remove "/*" within comment.
* win32/win32.c (set_pioinfo_extra): use more reliable way to search
the position of pioinfo of VC14, and also support debug library of it.
patched by davispuh AT gmail.com
[ruby-core:76644] [Bug #12644]
this fixes also [Bug #12631]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62884 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
r62810. ruby_2_3 does not output "[IMPORTANT]" message.
see [Backport #13566]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix size on Enumerable#cycle when the size is 0 [Bug #14082].
Patch by Kenichi Kamiya
test/ruby/test_lazy_enumerator.rb: test for [Bug #14082]
enum.c: check argument first
* enum.c (enum_cycle_size): check an argument before the size of
the receiver, if it is given.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62827 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_here_document): an escaped newline is not an
actual newline, and the rest part should not be dedented.
[ruby-core:72855] [Bug #11989]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62826 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
thread_pthread: do not corrupt stack
This fixes stuck test/ruby/test_io.rb with FIBER_USE_NATIVE=0 on
GNU/Linux because linked-list pointers used by glibc get
corrupted when fiber stacks are copied.
Thanks to wanabe for finding the bug and original patch.
* thread_pthread (native_thread_init_stack): fix stack corruption
[ruby-core:82737] [Bug #13387]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62825 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Clarify the behavior of IO.write without offset in write mode.
https://github.com/ruby/ruby/pull/1571
Patch by @takanabe [fix GH-1571]
[Bug #11638][ruby-core:71277]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
parse.y: token info with BOM
* parse.y (parser_prepare): set token_info_enabled flag first,
before returning at BOM. [ruby-dev:50288] [Bug #13998]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
io.c: fix segfault with closing socket on Windows
* io.c (fptr_finalize_flush): add an argument to keep GVL.
* io.c (fptr_finalize): adjust for above change.
* io.c (io_close_fptr): closing without GVL causes another
exception while raising exception in another thread. This causes
segfault on Windows. Keep GVL while closing when another thread
raises.
[Bug #13856] [ruby-core:82602]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62821 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/forwardable.rb (_delegator_method): leave the backtrace
untouched during accessor. forwardable.rb does not appear in
the backtrace during delegated method because of tail-call
optimization.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62820 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
dup String#partition return value
* string.c (rb_str_partition): return duplicated receiver, when no
splits. [ruby-core:82911] [Bug#13925]
Author: Seiei Miyagi <hanachin@gmail.com>
dup String#rpartition return value
* string.c (rb_str_rpartition): return duplicated receiver, when
no splits. [ruby-core:82911] [Bug#13925]
Author: Seiei Miyagi <hanachin@gmail.com>
dup String#split return value
* string.c (rb_str_split): return duplicated receiver, when no
splits. patched by tompng (tomoya ishida) in [ruby-core:82911],
and the test case by Seiei Miyagi <hanachin@gmail.com>.
[Bug#13925] [Fix GH-1705]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62818 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ruby.c: paragraph mode by -00
* ruby.c (proc_options): set to paragraph mode, if -00 is given,
as well as perl and -R0 option in 0.49.
[ruby-core:81987] [Bug #13736]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62817 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
process: block/unblock signals around fork
As with forking for execve(2) in `spawn', we must block signals
to ensure they are handled correctly in a freshly `fork'-ed child.
* process.c (retry_fork_ruby): block/unblock signals around fork
(rb_fork_ruby): re-enable signals in forked child
* test/ruby/test_process.rb (test_forked_child_signal): new test
[ruby-core:82883] [Bug #13916]
Thanks to Russell Davis for the bug report and test case.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62816 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
vm.c: fix `cfp consistency error' which occurs
when raising exception in bmethod call event
* vm.c (invoke_bmethod): set FINISH flag just before calling vm_exec.
[ruby-dev:50162] [Bug #13705]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62815 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
High Sierra's APFS cannot use invalid filenames [Bug #13816]
added workaround for APFS file format.
* TestFileExhaustive#test_atime: It fails with nano-sec precise.
I changed to use unixtime for this assertion for APFS.
* TestFileExhaustive#test_expand_path: skip assertion when given
invalid charactor on APFS.
[Bug #13816][ruby-core:82383]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
test_framework.rb: for case-sensitive filesystem
* test/mkmf/test_framework.rb (test_single_framework): fix header
file name for case-sensitive filesystem. it may not be same as
the framework name, but should be the actual file name.
[ruby-dev:50093] [Bug #13489]
* test/mkmf/test_framework.rb (test_multi_frameworks): ditto.
test_file_exhaustive.rb: check case-sensitive fs
* test/ruby/test_file_exhaustive.rb (test_expand_path): dump
expanded file name, not only appended char, for case-sensitive
filesystem. [ruby-dev:50093] [Bug #13489]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62813 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
test/win32ole/test_word.rb: word quit without confirmation dialog to save
files. [Bug #13894] Thanks to h.shirosaki.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62812 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
fix the case High Sierra's mincore(2) may return -128 [Bug #13895]
Fix typos [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62811 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Treat NULL reference case [Bug #13566]
Fix C level backtrace on Darwin
SEGV caused by invalid instruction call.
skip unless PLATFORM is darwin.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.in (ARFLAGS): check if deterministic mode flag is
effective, which is on by default on Ubuntu.
configure.in: use libtool
* configure.in: use libtool on macOS to suppress a warning against
debug_counter.o, which has no symbols unless USE_DEBUG_COUNTER
is set to non-zero.
configure.in: use newer libtool only
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@62809 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
