Commit graph

69 commits

Author SHA1 Message Date
Kazuhiro NISHIYAMA
3238641750
Use filesystem encoding for file path only
`path_info` contains filesystem encoding and binary.
Example is `"/webrick.cgi/%A5%DB%A4%B2/%A4%DB%A4%B2"` in `TestWEBrickCGI#test_cgi`.
2020-06-22 17:37:37 +09:00
Kazuhiro NISHIYAMA
32e2440a66
Fix ERROR ArgumentError: invalid byte sequence in UTF-8
http://ci.rvm.jp/results/trunk-test@ruby-sky1/3012894
```
/tmp/ruby/v3/src/trunk-test/tool/lib/minitest/unit.rb:199:in `assert': webrick log start: (MiniTest::Assertion)
  [2020-06-19 23:01:59] ERROR ArgumentError: invalid byte sequence in UTF-8
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpservlet/filehandler.rb:336:in `scan'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpservlet/filehandler.rb:336:in `set_filename'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpservlet/filehandler.rb:310:in `exec_handler'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpservlet/filehandler.rb:245:in `do_GET'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpservlet/abstract.rb:105:in `service'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpservlet/filehandler.rb:241:in `service'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpserver.rb:140:in `service'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/httpserver.rb:96:in `run'
  	/tmp/ruby/v3/src/trunk-test/lib/webrick/server.rb:307:in `block in start_thread'
```
2020-06-19 23:06:38 +09:00
Kazuhiro NISHIYAMA
97c1782db6
Use filesystem encoding as FileHandler's encoding
instead of `@root.encoding`.
And fallback to ASCII-8BIT when filesystem encoding is US-ASCII.

When `@root.encoding` is not compatible filesystem encoding,
`Encoding::CompatibilityError` raised at `webrick/httpservlet/filehandler.rb:341`.
So `DocumentRoot` must be compatible with filesystem encoding.
2020-06-19 22:56:29 +09:00
Kazuhiro NISHIYAMA
78d4eace02
Do not change local_path encoding in WEBrick::HTTPServlet::DefaultFileHandler
This reverts 750203c514 and 93e6fa1d31
2020-06-19 22:08:19 +09:00
Kazuhiro NISHIYAMA
6fe1919486
Fix failure on mswin CI
20200619T054159Z.fail.html.gz
```
  1) Failure:
WEBrick::TestFileHandler#test_cjk_in_path [D:/tmp/mswin-build20200619-14304-utgij/ruby/test/webrick/utils.rb:72]:
exceptions on 2 threads:
webrick log start:
  [2020-06-19 16:28:42] ERROR `/あ.txt' not found.

webrick log end
Filesystem encoding is Windows-31J.
<"200"> expected but was
<"404">.
---
<[]> expected but was
<["[2020-06-19 16:28:42] ERROR `/\xE3\x81\x82.txt' not found.\n"]>.
```

`prevent_directory_traversal` treats `path_info` as filesystem encoding.
So path_info should be filesystem encoding in request URL.

On some environments, fallback to ASCII-8BIT when EncodingError.
2020-06-19 20:13:22 +09:00
Kazuhiro NISHIYAMA
daa9dcd57e
Hide error location from error message 2020-06-19 20:13:22 +09:00
Yusuke Endoh
750203c514 lib/webrick/httpservlet/filehandler.rb: always handle a path as UTF-8
20200618T113134Z.fail.html.gz
```
  1) Failure:
WEBrick::TestFileHandler#test_cjk_in_path [D:/tmp/mswin-build20200618-84004-1t0dh8f/ruby/test/webrick/utils.rb:72]:
exceptions on 2 threads:
webrick log start:
  [2020-06-18 22:18:07] ERROR `/??.txt' not found.

webrick log end
Filesystem encoding is Windows-31J.
<"200"> expected but was
<"404">.
```
2020-06-18 23:43:48 +09:00
Kazuhiro NISHIYAMA
93e6fa1d31 Use filesystem encoding in do_GET of filehandler
Try to fix 404 error on mswinci.

20200614T225859Z.fail.html.gz
```
  1) Failure:
WEBrick::TestFileHandler#test_cjk_in_path [D:/tmp/mswin-build20200615-24932-11ykstf/ruby/test/webrick/utils.rb:72]:
exceptions on 2 threads:
webrick log start:
  [2020-06-15 09:48:29] ERROR `/あ.txt' not found.

webrick log end.
<"200"> expected but was
<"404">.
---
<[]> expected but was
<["[2020-06-15 09:48:29] ERROR `/\xE3\x81\x82.txt' not found.\n"]>.
```
2020-06-15 11:25:33 +09:00
Kazuhiro NISHIYAMA
e698bf1a7f [ruby/webrick] Fix httpd error in CJK directory
[Bug #16753]

83cf440858
2020-06-11 13:38:32 +09:00
Nobuyoshi Nakada
e68999c82c
Fixed misspellings
Fixed misspellings reported at [Bug #16437], for default gems.
2019-12-20 12:19:45 +09:00
Jeremy Evans
c75100d004 [ruby/webrick] Allow WEBrick::HTTPServlet::CGIHandler :CGIInterpreter option to be array
This way you don't need to escape each entry.

Implements Ruby Feature 15170.

d8086e600c
2019-11-30 17:48:15 +09:00
mame
7a635a7d12 lib/webrick: explicitly convert header values to a string
The values of @header are expected to be all strings;
WEBrick::HTTPResponse::[]=(key, val) explicitly converts the second
argument to a string and assigns it to @header hash.
However, there were some points in WEBrick internal code that assigns
non-String to @header.  This change fixes the issues.

The values are checked by `header_value =~ /\r\n/` in check_header.
The type confusion caused conflict with removal of `Object#=~`
[Feature #15231].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 05:29:45 +00:00
marcandre
b9d42af0f2 lib/*: Prefer require_relative over require, remove explicit extension
[#15206] [Fix GH-1976]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65506 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-02 17:52:43 +00:00
marcandre
e859e668d2 lib/*: Prefer require_relative over require.
[#15206] [Fix GH-1976]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65505 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-02 17:52:33 +00:00
usa
c1718e988e get rid of test error/failure on Windows introduced at r62955
* lib/webrick/httpresponse.rb (send_body_io): use seek if NotImplementedError
  is raised in IO.copy_stream with offset.

* lib/webrick/httpservlet/filehandler.rb (multipart_body): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 13:27:35 +00:00
normal
89450a80fc webrick/httpservlet/cgihandler: reduce memory use
WEBrick::HTTPRequest#body can be passed a block to process the
body in chunks.  Use this feature to avoid building a giant
string in memory.

* lib/webrick/httpservlet/cgihandler.rb (do_GET):
  avoid reading entire request body into memory
  (do_POST is aliased to do_GET, so it handles bodies)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62961 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:28 +00:00
normal
4639ac8953 webrick: use IO.copy_stream for multipart response
Use the new Proc response body feature to generate a multipart
range response dynamically.  We use a flat array to minimize
object overhead as much as possible; as many ranges may fit
into an HTTP request header.

* lib/webrick/httpservlet/filehandler.rb (multipart_body): new method
  (make_partial_content): use multipart_body

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62959 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:18 +00:00
normal
6676a217f3 webrick: quiet warning for multi-part ranges
Content-Length is ignored by WEBrick::HTTPResponse even if we
calculate it, so instead we chunk responses to HTTP/1.1 clients
and terminate HTTP/1.0 connections.

* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
  quiet warning

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62957 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:06:08 +00:00
normal
6360243fd2 webrick: use IO.copy_stream for single range response
This is also compatible with range responses generated
by Rack::File (tested with rack 2.0.3).

* lib/webrick/httpresponse.rb (send_body_io): use Content-Range
* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
  use File object for the single range case
* test/webrick/test_filehandler.rb (get_res_body): use send_body
  to test result

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 08:05:57 +00:00
normal
0d92f156b6 webrick/httpservelet/cgi_runner: avoid IO#reopen on pathname
IO#reopen seems to behave strangely on Win32 based on the logs
posted by Greg for [Bug #14420].  Lets try to fix the issue by
using File.open explicitly (but not Kernel#open).

* lib/webrick/httpservelet/cgi_runner: use File.open explicitly
  [Bug #14220]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61409 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22 05:59:11 +00:00
normal
1ad355bd53 webrick/httpservlet/*handler: use File.open
This makes future code audits easier.  None of these changes
fix realistic remote code execution vulnerabilities because
we stat(2) before attempting Kernel#open.

* lib/webrick/httpservlet/erbhandler.rb (do_GET): use File.open
* lib/webrick/httpservlet/filehandler.rb (do_GET): use File.open
  (make_partial_content): ditto
  [Misc #14216]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61401 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22 01:08:00 +00:00
normal
646b83af2a webrick/httpservlet/cgi_runner.rb: remove unnecessary open
IO#reopen already takes string path names as well as IO objects
(but not "| command" strings)

This makes further auditing for inadvertant code execution
easier.  There's no actual bugfix or behavior change here,
as no external data is passed to cgi_runner.rb.

* lib/webrick/httpservlet/cgi_runner.rb: remove Kernel#open call
  [Misc #14216]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61398 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22 01:07:45 +00:00
kazu
6064132c42 Remove unnecessary require 'thread'
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60139 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-08 07:00:01 +00:00
normal
dafeebf12d webrick: filter out HTTP_PROXY for CGIHandler
* lib/webrick/httpservlet/cgihandler.rb (do_GET): delete HTTP_PROXY
* test/webrick/test_cgi.rb (test_cgi_env): new test
* test/webrick/webrick.cgi (do_GET): new endpoint to dump env
  [ruby-core:76511] [Bug #12610]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55731 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-22 16:43:12 +00:00
hsbt
65793c9c17 * lib/webrick/httpservlet/filehandler.rb: fix documentation for namespace.
[fix GH-1219][ci skip] Patch by @leafac

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53648 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-01-25 00:33:17 +00:00
naruse
3e92b635fb Add frozen_string_literal: false for all files
When you change this to true, you may need to add more tests.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-16 05:07:31 +00:00
hsbt
c9134128cb * lib/webrick/httpproxy.rb: remvoe useless assigned variables.
* lib/webrick/httpservlet/cgihandler.rb: ditto.
* lib/webrick/httpservlet/erbhandler.rb: ditto.
* lib/webrick/server.rb: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46560 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-26 09:05:58 +00:00
nobu
14922c7033 httpservlet/filehandler.rb: pass queries
* lib/webrick/httpservlet/filehandler.rb (set_dir_list): pass the
  given queries to subdirectories.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44919 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-02-12 09:01:30 +00:00
nobu
8fb4108bb7 * lib/webrick/httpservlet/filehandler.rb (set_dir_list): add NameWidth option.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 13:41:21 +00:00
nobu
31c2e85e51 * lib/webrick/httpservlet/filehandler.rb (set_dir_list): use TABLE.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40861 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 13:41:19 +00:00
nobu
b6f42822fd * lib/webrick/httpservlet/filehandler.rb (set_dir_list): set charset.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40860 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 13:41:16 +00:00
nobu
83fa035d24 * lib/webrick/httpservlet/filehandler.rb (set_dir_list): share title and h1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40859 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 13:41:10 +00:00
nobu
bcddf03238 filehandler.rb: revert r20152
* lib/webrick/httpservlet/filehandler.rb (set_dir_list): revert r20152
  partially and fix misuse of bytesize and regexp repetition operator.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40847 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 00:53:40 +00:00
drbrain
28afe277a8 * lib/webrick/accesslog.rb: Improved WEBrick documentation.
* lib/webrick/cgi.rb:  ditto.
* lib/webrick/config.rb:  ditto.
* lib/webrick/cookie.rb:  ditto.
* lib/webrick/httpauth/authenticator.rb:  ditto.
* lib/webrick/httpauth/basicauth.rb:  ditto.
* lib/webrick/httpauth/digestauth.rb:  ditto.
* lib/webrick/httpproxy.rb:  ditto.
* lib/webrick/httprequest.rb:  ditto.
* lib/webrick/httpresponse.rb:  ditto.
* lib/webrick/https.rb:  ditto.
* lib/webrick/httpserver.rb:  ditto.
* lib/webrick/httpservlet/cgihandler.rb:  ditto.
* lib/webrick/httpservlet/filehandler.rb:  ditto.
* lib/webrick/httpservlet/prochandler.rb:  ditto.
* lib/webrick/httputils.rb:  ditto.
* lib/webrick/httpversion.rb:  ditto.
* lib/webrick/log.rb:  ditto.
* lib/webrick/server.rb:  ditto.
* lib/webrick/ssl.rb:  ditto.
* lib/webrick/utils.rb:  ditto.
* lib/webrick/version.rb:  ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@38945 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-01-26 01:12:54 +00:00
zzak
902a36c1f6 * lib/webrick/httpservlet/abstract.rb (WEBrick::HTTPServlet): Typo in
example. Patch by shlensky [Fixes #232 on github]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@38709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-01-05 17:24:40 +00:00
drbrain
35bb53430c * lib/webrick/httpservlet/erbhandler.rb: Allow the ERB document to
alter the content-type of the response.  [Ruby 1.9 - Bug #4685]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@32117 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-06-16 05:37:31 +00:00
nobu
3a47cf3395 * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-05-15 11:55:52 +00:00
drbrain
d6e3bc4ce4 * lib/webrick/httpservlet/erbhandler.rb: Add documentation.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31540 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-05-12 22:37:01 +00:00
drbrain
071a678a15 * lib/webrick: Add Documentation
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-05-10 00:13:58 +00:00
marcandre
f3e3547407 * lib/*.rb: Remove unused variable warnings.
Patch by Run Paint [ruby-core:30991]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29726 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-11-08 20:59:01 +00:00
mame
ab408009fc * lib/webrick/httpservlet/filehandler.rb
(prevent_directory_traversal): apply filesystem encoding to path
  only during calling File.expand_path.  [ruby-dev:41423]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-06-17 16:26:00 +00:00
naruse
d7cc39658c * lib/webrick/httpservlet/cgihandler.rb
(WEBrick::HTTPServlet::CGIHandler#do_GET):
  set binary mode for tempfile.
  1265467681/286

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26887 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-03-12 08:48:10 +00:00
nobu
f00951bb25 * lib/webrick/httpservlet/filehandler.rb (make_partial_content):
add bytes-unit.  [ruby-dev:40030]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26346 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-01-18 08:16:07 +00:00
akr
c0b4e90f08 * tool/mkconfig.rb: generate RbConfig.ruby in rbconfig.rb.
* lib/webrick/httpservlet/cgihandler.rb: use RbConfig.ruby.

* test/ruby/envutil.rb: ditto.

* benchmark/report.rb: ditto.

* benchmark/runc.rb: ditto.

* tool/eval.rb: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26216 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-12-31 15:00:04 +00:00
naruse
604f01f2b8 * lib/webrick/httpservlet/filehandler.rb: escape filename of index.
[ruby-dev:37768]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26026 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-12-06 07:04:50 +00:00
nobu
c0a09d9594 * lib/webrick/httpservlet/abstract.rb (do_OPTIONS): method names
are symbols now.  [ruby-core:24580]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24742 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-03 19:46:18 +00:00
nobu
287a34ae0d * {ext,lib,test}/**/*.rb: removed trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-03-06 03:56:38 +00:00
matz
877ac7236a * lib/webrick/httpproxy.rb (WEBrick::HTTPProxyServer#do_CONNECT):
use #bytesize instead of #size.  a patch submitted from
  raspberry lemon in  [ruby-core:18571].

* lib/webrick/httpauth/digestauth.rb, lib/webrick/httpproxy.rb,
  lib/webrick/httprequest.rb, lib/webrick/httpservlet/cgi_runner.rb,
  lib/webrick/httpservlet/abstract.rb, lib/webrick/httpresponse.rb,
  lib/webrick/httpservlet/cgihandler.rb, lib/webrick/utils.rb: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@20152 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-11-08 09:41:24 +00:00
knu
d27c31e041 * lib/webrick/httpservlet/cgihandler.rb (WEBrick::HTTPServlet::CGIHandler#do_GET):
Set the HTTP status code to 302 if a Location header field is
  present and the status code is not valid as a client
  redirection.  cf. RFC 3875 6.2.3, 6.2.4.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-05-28 07:13:04 +00:00
gotoyuzo
8ee3267d26 * lib/webrick/httpservlet/filehandler.rb: should normalize path
name in path_info to prevent script disclosure vulnerability on
  DOSISH filesystems. (fix: CVE-2008-1891)
  Note: NTFS/FAT filesystem should not be published by the platforms
  other than Windows. Pathname interpretation (including short
  filename) is less than perfect.

* lib/webrick/httpservlet/abstract.rb
  (WEBrick::HTTPServlet::AbstracServlet#redirect_to_directory_uri):
  should escape the value of Location: header.

* lib/webrick/httpservlet/cgi_runner.rb: accept interpreter
  command line arguments.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16453 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-05-18 13:33:24 +00:00