Commit graph

11281 commits

Author SHA1 Message Date
kazu
d0775a1120 test/lib/leakchecker.rb: Add check_verbose
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-31 02:29:19 +00:00
k0kubun
378c0bcd4f test_jit.rb: loosen CC check condition
see also: r63037. Probably it was not enough.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63042 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-30 15:08:52 +00:00
nobu
484cc6b3bc compile.c: do not dump unused callinfos
* compile.c (compile_if): rewind callinfo indexes used in
  unreachable paths, to get rid of dumping unused callinfos.
  [ruby-core:86399] [Bug #14553]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63040 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-30 05:15:07 +00:00
k0kubun
46b391ff73 test_jit.rb: loosen debug log check
We couldn't catch log with
https://gist.github.com/ko1/0fdc6a8d4add70cd8648000b16a3da38#file-brlog-trunk_gcc6-20180329-003723-L370
because the error message started with "gcc-6:".

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-29 12:37:31 +00:00
nobu
9c5d90b516 Use only CMDARG/COND _PUSH/POP for cmdarg/cond management.
From: Ilya Bylich <ibylich@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63033 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-29 03:42:32 +00:00
nobu
e42f4ae773 array.c: yield blockarg in collect
* array.c (rb_ary_collect): yield in block argument semantics
  always to splat array elements to lambda, for the backward
  compatibility.  [ruby-core:86362] [Bug #14639]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63030 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-29 00:05:45 +00:00
nobu
d02b7bd864 pack.c: fix underflow
* pack.c (pack_unpack_internal): get rid of underflow.
  https://hackerone.com/reports/298246

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62992 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 10:12:17 +00:00
nobu
8794dec6a5 unixsocket.c: check NUL bytes
* ext/socket/unixsocket.c (rsock_init_unixsock): check NUL bytes.
  https://hackerone.com/reports/302997

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62991 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 10:08:57 +00:00
hsbt
10b96900b9 Ignore file separator from tmpfile/tmpdir name.
From: SHIBATA Hiroshi <hsbt@ruby-lang.org>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62990 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 10:03:03 +00:00
nobu
bd5661a3cb dir.c: check NUL bytes
* dir.c (GlobPathValue): should be used in rb_push_glob only.
  other methods should use FilePathValue.
  https://hackerone.com/reports/302338

* dir.c (rb_push_glob): expand GlobPathValue

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62989 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 09:58:52 +00:00
nobu
22a4e6ac7a adjust indents [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62969 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:48:52 +00:00
normal
d9d4a28f1c webrick: prevent response splitting and header injection
Original patch by tenderlove (with minor style adjustments).

* lib/webrick/httpresponse.rb (send_header): call check_header
  (check_header): raise on embedded CRLF in header value
* test/webrick/test_httpresponse.rb
  (test_prevent_response_splitting_headers): new test
* (test_prevent_response_splitting_cookie_headers): ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62968 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:38:26 +00:00
normal
7292569d3f io.c: IO#write without args returns 0
This is consistent with other implementations of .write
in openssl and stringio.

* io.c (io_write_m): return 0 on argc == 0
  [ruby-core:86285] [Bug #14338]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:38:21 +00:00
normal
706c028909 webrick/httpproxy: stream request and response bodies
Reading entire request or response bodies into memory can lead
to trivial denial-of-service attacks.  Introduce Fibers in both
cases to allow streaming.

WEBrick::HTTPRequest gains a new body_reader method to prepare
itself as a source for IO.copy_stream.  This allows the
WEBrick::HTTPRequest object to be used as the
Net::HTTPGenericRequest#body_stream= arg for Net::HTTP.

For HTTP proxy response bodies, we also use a Fiber to
to make the HTTP request and read the response body.

* lib/webrick/httprequest.rb (body_reader): new method
  (readpartial): ditto
* lib/webrick/httpproxy.rb (perform_proxy_request): use Fiber
  to stream response body
  (do_GET, do_HEAD): adjust call
  (do_POST): adjust call and supply body_reader
* test/webrick/test_httprequest.rb (test_chunked): test
  for IO.copy_stream compatibility
* test/webrick/test_httpproxy.rb (test_big_bodies): new test

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:55 +00:00
normal
df2ce3434f webrick: add test for Digest auth-int
No changes to the actual code, this is a new test for
a feature for which no tests existed.  I don't understand
the Digest authentication code well at all, but this is
necessary for the subsequent change.

* test/webrick/test_httpauth.rb (test_digest_auth_int): new test
  (credentials_for_request): support bodies with POST

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62964 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:44 +00:00
normal
2d41d88c4d webrick/httprequest: use InputBufferSize for chunked requests
While WEBrick::HTTPRequest#body provides a Proc interface
for streaming large request bodies, clients must not force
the server to use an excessively large chunk size.

* lib/webrick/httprequest.rb (read_chunk_size): limit each
  read and block.call to :InputBufferSize in config.
* test/webrick/test_httpserver.rb (test_big_chunks): new test

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62963 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:39 +00:00
normal
6db6eb572b webrick/httprequest: raise correct exception
"BadRequest" alone does not resolve correctly, it is in the
HTTPStatus namespace.

* lib/webrick/httprequest.rb (read_chunked): use correct exception
* test/webrick/test_httpserver.rb (test_eof_in_chunk): new test

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:34 +00:00
normal
7e1f2a5aa9 webrick/httprequest: limit request headers size
We use the same 112 KB limit started (AFAIK) by Mongrel, Thin,
and Puma to prevent malicious users from using up all the memory
with a single request.  This also limits the damage done by
excessive ranges in multipart Range: requests.

Due to the way we rely on IO#gets and the desire to keep
the code simple, the actual maximum header may be 4093 bytes
larger than 112 KB, but we're splitting hairs at that point.

* lib/webrick/httprequest.rb: define MAX_HEADER_LENGTH
  (read_header): raise when headers exceed max length

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62960 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:23 +00:00
normal
0478a37903 test/webrick/test_filehandler.rb: stricter multipart range test
We need to ensure we generate compatibile output in
the face of future changes

* test/webrick/test_filehandler.rb (test_make_partial_content):
  check response body

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62956 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:02 +00:00
normal
6360243fd2 webrick: use IO.copy_stream for single range response
This is also compatible with range responses generated
by Rack::File (tested with rack 2.0.3).

* lib/webrick/httpresponse.rb (send_body_io): use Content-Range
* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
  use File object for the single range case
* test/webrick/test_filehandler.rb (get_res_body): use send_body
  to test result

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:05:57 +00:00
nobu
ebf5348528 win32/file.c: relative path with drive letter
* win32/file.c (IS_ABSOLUTE_PATH_P): home directory should not be
  a relative path regardless a drive letter.   PathIsRelativeW
  returns FALSE on such path.  [ruby-core:86356] [Bug #14638]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62952 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 07:01:48 +00:00
normal
a2d63ea2fb thread_sync.c: avoid reaching across stacks of dead threads
rb_ensure is insufficient cleanup for fork and we must
reinitialize all waitqueues in the child process.

Unfortunately this increases the footprint of ConditionVariable,
Queue and SizedQueue by 8 bytes on 32-bit (16 bytes on 64-bit).

[ruby-core:86316] [Bug #14634]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62934 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-27 09:28:37 +00:00
kazu
ec74ad7cba Fix test error
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62926 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-26 10:18:46 +00:00
hsbt
98c7058bf7 Merge RDoc 6.0.3 from upstream.
It fixed the several bugs that was found after RDoc 6 releasing.

From: SHIBATA Hiroshi <hsbt@ruby-lang.org>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62924 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-26 05:56:26 +00:00
nobu
f28d6165f6 assertions for r62914
* test/ruby/test_keyword.rb (test_splat_hash): assertion on
  mandatory and rest arguments.  r62914 is not only for optional
  argument.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62919 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-25 14:06:06 +00:00
nobu
a23eca2672 vm_args.c: warn splat to var
* vm_args.c (setup_parameters_complex): [EXPERIMENTAL] warn when
  splat keyword arguments is passed as a single ordinary argument,
  not as a keyword rest argument.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-25 02:22:14 +00:00
nobu
f3fbf1dfad parse.y: k_else in bodystmt
* parse.y (bodystmt): expand opt_else to show the error message at
  the right place.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62902 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-23 01:09:57 +00:00
nobu
140512d222 parse.y: else without rescue
* parse.y (bodystmt): [EXPERIMENTAL] make `else` without `rescue`
  a syntax error.  [DevelopersMeeting20180315Japan]

  https://bugs.ruby-lang.org/projects/ruby/wiki/DevelopersMeeting20180315Japan
  https://docs.google.com/document/d/1RT0ijSo8uJ4Awn3CEvuYkjH0TVeXSYgeAFNmVGYC3ak/edit#
  > * do-else-end
  >   https://twitter.com/joker1007/status/974173396006129664
  >   https://twitter.com/joker1007/status/974173641347756032
  >   https://twitter.com/joker1007/status/974176512554369027
  >   Will be SyntaxError in 2.6-preview2
  >   All of begin/do/def (experimental)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62901 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-23 00:40:08 +00:00
nobu
e852f9d624 test_exception.rb: assertions with Exception.to_tty?
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62899 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 16:05:04 +00:00
naruse
234a30459c set UTF-8 if given URI string is ASCII
Now URI is normally UTF-8, and US-ASCII URI string is considered as
escaped a UTF-8 string.
https://github.com/rails/rails/issues/32294

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62897 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 11:20:03 +00:00
nobu
b9881083f1 error.c: full_message options
* error.c (exc_full_message): add highlight: and reverse: keyword
  options.  [Bug #14324]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62894 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 08:26:23 +00:00
naruse
41b2ef4685 fix each_grapheme_cluster's size [Bug #14363]
From: Hugo Peixoto <hugo.peixoto@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 07:58:38 +00:00
naruse
6e0f5b8407 Revert "each_grapheme_cluster shouldn't return size [Bug #14363]"
This reverts commit r62887.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62891 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 07:58:37 +00:00
naruse
613decd088 each_grapheme_cluster shouldn't return size [Bug #14363]
From: Stefan Schüßler <mail@stefanschuessler.de>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62888 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 06:59:54 +00:00
nobu
b6e2c52857 Docs and tests on URI.hierarchical?, URI.absolute?
Improve code coverage and clarify meaning of hierarchical based on RFC
text.

[Fix GH-1846]

From: Xavier Riley <xavriley@hotmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62882 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-21 16:03:59 +00:00
nobu
58fbe69a5b parse.y: terminator at continued line
* parse.y (here_document): a continuing line is not the
  terminator.  [ruby-core:86283] [Bug #14621]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62873 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-21 12:40:16 +00:00
nobu
491f523ab1 parse.y: unindent continued line
* parse.y (tokadd_string): stop at continued line in dedented here
  documents, to dedent for each lines before removing escaped
  newlines.  [ruby-core:86236] [Bug #14621]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-21 10:32:15 +00:00
nobu
bc7e5fcb32 test_iseq.rb: disable load_from_binary
* test/ruby/test_iseq.rb (test_to_binary_with_objects): disable
  load_from_binary right now, r62851 seems having a trouble around
  get_insn_info_succinct_bitvector() yet.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62871 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-21 10:06:23 +00:00
shugo
798316eac2 io.c: Methods of File should not invoke external commands
For security reasons, File.read, File.binread, File.write, File.binwrite,
File.foreach, and File.readlines should not invoke external commands even
if the path starts with the pipe character |.
[ruby-core:84495] [Feature #14245]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62857 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-20 09:09:49 +00:00
nobu
ce848356ae compile.c: fix ibf_load_code
* compile.c (ibf_load_iseq_each): manage iseq_size to point loaded
  objects in iseq_encoded.  now marking iseq scans iseq_encoded
  directly.

* test/ruby/test_iseq.rb (test_to_binary_with_objects): skip for
  now, but fix argument order of assert_equal.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-20 08:36:42 +00:00
nobu
d34bc779a7 parse.y: mismatched indentations at middle
* parse.y (k_rescue, k_ensure, k_when, k_else, k_elsif): warn
  mismatched indentations at keywords in middle of blocks.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62836 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-19 08:21:26 +00:00
nobu
9720136d9c compile.c: arg var index
* compile.c (iseq_set_arguments): determine argument variable
  indexes by the order, not by just IDs.  arguments begin with `_`
  can be duplicate, so by-ID index may result in a wrong value.
  [ruby-core:86159] [Bug #14611]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62833 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-19 04:08:19 +00:00
nobu
8bad3e6b9d IBF works only on x86 platforms now
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62800 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-18 03:40:49 +00:00
k0kubun
f5fc032eee Remove unused variables
This fixes following warning:

```
test/logger/test_logger.rb:329: warning: assigned but unused variable - read_read
```

[Fix GH-1840]

From: yuuji.yaginuma <yuuji.yaginuma@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62794 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-17 06:41:35 +00:00
k0kubun
4b3809114b test_jit.rb: debug cc1 availability
ENV didn't have any useful information.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-17 06:33:28 +00:00
nobu
6cd5db8904 test_weakmap.rb: skip unstable assertion
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62781 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-16 14:12:39 +00:00
nobu
8a8f542c43 re.c: do not escape terminator in Regexp.union
* re.c (rb_reg_str_with_term): change terminator.

* re.c (rb_reg_s_union): terminator in source string does not need
  to be escaped.  terminators are outside of regexp source itself.
  [ruby-core:86149] [Bug #14608]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62779 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-16 13:37:44 +00:00
nobu
1d07bb84f8 test_iseq.rb: skip iseq with coverage
* test/ruby/test_iseq.rb (test_to_binary_with_objects): #to_binary
  does not support iseq compiled with coverage, just skip.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-16 12:06:55 +00:00
nobu
ddf295a26f compile.c: fix load_from_binary
* compile.c (ibf_load_iseq_each): realpath may be nil.  follow up
  r59709.  [fix https://github.com/Shopify/bootsnap/issues/132]

From: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62772 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-16 02:27:50 +00:00
naruse
0129fc8f58 Revert r62617 "compile.c: fix load_from_binary"
It breaks Solaris:
20180301T012502Z.diff.html.gz

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62769 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-15 16:51:34 +00:00