Commit graph

45 commits

Author SHA1 Message Date
Nobuyoshi Nakada
5b98b2ce39 win32: Use UTF-8 as filesystem encoding [Feature #12654]
Co-Authored-By: Dāvis Mosāns <davispuh@gmail.com>
2020-12-20 18:34:04 +09:00
Jeremy Evans
80b5a0ff2a
Make rb_scan_args handle keywords more similar to Ruby methods (#2460)
Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.

This makes the following changes to : handling.

* Treats as **kw, prompting keyword argument separation warnings
  if called with a positional hash.

* Do not look for an option hash if empty keywords are provided.
  For backwards compatibility, treat an empty keyword splat as a empty
  mandatory positional hash argument, but emit a a warning, as this
  behavior will be removed in Ruby 3.  The argument number check
  needs to be moved lower so it can correctly handle an empty
  positional argument being added.

* If the last argument is nil and it is necessary to treat it as an option
  hash in order to make sure all arguments are processed, continue to
  treat the last argument as the option hash. Emit a warning in this case,
  as this behavior will be removed in Ruby 3.

* If splitting the keyword hash into two hashes, issue a warning, as we
  will not be splitting hashes in Ruby 3.

* If the keyword argument is required to fill a mandatory positional
  argument, continue to do so, but emit a warning as this behavior will
  be going away in Ruby 3.

* If keyword arguments are provided and the last argument is not a hash,
  that indicates something wrong. This can happen if a cfunc is calling
  rb_scan_args multiple times, and providing arguments that were not
  passed to it from Ruby.  Callers need to switch to the new
  rb_scan_args_kw function, which allows passing of whether keywords
  were provided.

This commit fixes all warnings caused by the changes above.

It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used.  If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.

In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby.  The last argument may or may not be a hash,
so we can't set keyword argument mode.  However, if it is a
hash, we don't want to warn when treating it as keywords.

In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args.  Also, make
sure not to pass nil in place of an option hash.

Work around Kernel#warn warnings due to problems in the Rubygems
override of the method.  There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.

Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.
2019-09-25 11:18:49 -07:00
naruse
5c63fc682c High Sierra's APFS cannot use invalid filenames [Bug #13816]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59602 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-15 21:33:43 +00:00
nobu
a0b6e36609 test_dir_m17n.rb: read in filesystem encoding
* test/ruby/test_dir_m17n.rb (test_entries_compose): read in
  filesystem encoding instead of default external encoding on
  Windows too.  these two encodings may differ on some
  environments.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58765 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-17 00:28:01 +00:00
shugo
2411cab586 Don't encode to UTF-8 if it's unnecessary.
If the file system encoding is ISO-8851-1 or if the encoding of the target
string is invalid, don't encode to UTF-8.  [ruby-core:78271] [Bug#12972]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-11-23 04:40:40 +00:00
duerst
b67ead1452 * test/ruby/test_dir_m17n.rb: Skip tests with non-UTF-8 encodings
on cygwin. Cygwin can use the Unicode PUA (private use area) to store
  bytes from non-UTF-8 filenames (see
  https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-specialchars),
  but we are not supporting this. [Bug #12443]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55453 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-19 07:55:25 +00:00
nobu
ab86bab950 test_dir_m17n.rb: fix ASCII-8BIT env
* test/ruby/test_dir_m17n.rb (test_glob_encoding): get rid of
  conversion when LC_ALL=C and filesystem encoding is ASCII-8BIT.
  [ruby-core:73868] [Bug #12081]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-02-20 09:36:08 +00:00
nobu
4f969f6eee dir.c: paths as UTF-8
* dir.c (push_pattern, push_glob): deal with read paths as UTF-8
  to stat later, on Windows as well as OS X.
  [ruby-core:73868] [Bug #12081]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53877 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-02-20 04:04:59 +00:00
naruse
3e92b635fb Add frozen_string_literal: false for all files
When you change this to true, you may need to add more tests.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-16 05:07:31 +00:00
nobu
c81b224edc test: use String#b instead of dup.force_encoding
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-14 06:40:55 +00:00
nobu
87a9dcf1b6 test_dir_m17n.rb: fix test
* test/ruby/test_dir_m17n.rb (test_pwd): fix test on some
  platforms.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52014 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-04 02:18:58 +00:00
nobu
c604000a31 dir.c: normalize cwd
* dir.c (rb_dir_getwd): normalize OS path to UTF-8 on OS X.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52012 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-04 01:10:00 +00:00
nobu
f4726dcdaa dir.c: same encoding to the pattern
* dir.c (push_pattern, push_glob): make globbed file names same
  encoding to the given pattern.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49744 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-25 06:59:39 +00:00
nobu
461c38a973 dir.c: fix escaping multibyte char
* dir.c (has_magic): fix escaping multibyte char, with glob
  meta-char in trailing bytes, e.g., Shift-JIS

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-07 00:11:00 +00:00
nobu
6548425ee0 test_dir_m17n.rb: fix tests
* test/ruby/test_dir_m17n.rb (with_enc_path): test with UTF-8
  default external encoding for non-locale environments.

* test/ruby/test_dir_m17n.rb (test_glob_warning_opendir): valid
  only if file permissions work.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49531 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-06 13:24:09 +00:00
nobu
ef5b8fe12f dir.c: preserve encoding
* dir.c (sys_enc_warning_in): preserve encoding of path name in
  warning messages.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49528 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-06 08:37:27 +00:00
akr
fb2008a73a * test/lib/envutil.rb: Moved from test/ruby/.
* test/lib/find_executable.rb: Ditto.

* test/lib/memory_status.rb: Ditto.

* test/lib/test/unit.rb: require envutil.

* test/: Don't require envutil in test files.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48409 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-13 16:05:37 +00:00
nobu
d85195712d test_dir_m17n.rb: assert_raw_file_name
* test/ruby/test_dir_m17n.rb (assert_raw_file_name): rename to
  show caller method in failure messages.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-28 04:20:35 +00:00
nobu
d551b78d4a test_dir_m17n.rb: fixed encoding file system on OS X
* test/ruby/test_dir_m17n.rb (test_filename_as_bytes_extutf8),
  (test_filename_bytes_euc_jp, test_filename_euc_jp),
  (test_filename_ext_euc_jp_and_int_utf_8): Mac OS X also has fixed
  encoding file system.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-08-14 00:10:58 +00:00
kou
9662e3ded5 * test/ruby/test_dir_m17n.rb (TestDir_M17N#test_entries_compose):
Use #each instead of #map just for iteration.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40880 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-21 12:50:21 +00:00
nobu
0fd2c7e67b test_dir_m17n.rb: sort Dir.entries
* test/ruby/test_dir_m17n.rb (test_entries_compose): the order on the
  filesystem does not matter, so sort Dir.entries result to compare.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-21 07:43:40 +00:00
naruse
32c5c675f2 force_encoding expected result as filesystem encoding
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-21 06:08:04 +00:00
nobu
1891b60f26 dir.c: compose HFS file names
* dir.c (dir_each): compose HFS file names from
  UTF8-MAC. [ruby-core:48745] [Bug #7267]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40868 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 18:11:23 +00:00
nobu
a7092270da test_dir_m17n.rb: assert_separately
* test/ruby/test_dir_m17n.rb: prefer assert_separately over
  assert_ruby_status for more descriptive messages.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40867 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-20 18:10:34 +00:00
nobu
29dc980e65 compose HFS file names
* dir.c (glob_helper): compose HFS file names from UTF8-MAC.
  [ruby-core:48745] [Bug #7267]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39821 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-03-19 08:06:53 +00:00
nobu
f1df537a83 test_dir_m17n.rb: test_glob_incompatible
* test/ruby/test_dir_m17n.rb (TestDir_M17N#test_glob_incompatible):
  move from test_dir.rb.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36904 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-09-05 03:03:24 +00:00
naruse
0daf636e7a Unix doesn't use win_expected_filename.
r36519's essensial part is not to use filename.force_encoding("euc-jp").

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36521 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-24 09:57:59 +00:00
nobu
cd1a6ab82b test_dir_m17n.rb: composite character
* test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_windows_1252_name):
  get rid of composite character for troublesome Mac OS X file system.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36520 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-24 09:36:14 +00:00
naruse
756b6e661b Add the case of Unix: filesystem encoding maybe US-ASCII.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36519 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-24 08:47:51 +00:00
usa
2b4f1eed4c * test/ruby/test_dir_m17n.rb: remove a garbage.
* test/ruby/test_dir_m17n.rb: convert from ascii-8bit to other encoding
  with 8bit bytes always fails.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36516 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-24 03:36:29 +00:00
usa
989a6cb176 * test/ruby/test_dir_m17n.rb: sorry, typo.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36515 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-24 03:32:55 +00:00
usa
3a4a494e02 * test/ruby/test_dir_m17n.rb: refactoring. RE should be in the left side
of the =~ operator, and compare the result with nil is meaningless.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36514 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-24 03:14:31 +00:00
luislavena
c4348fcd7f Fix raw UTF-8 test for non-Japanase Windows systems
* test/ruby/test_dir_m17n.rb (create_and_check_raw_file_name): add new
  helper method to ease encoding testing. Patch by Oleg Sukhodolsky.
  [ruby-core:46589][Bug #6765]

* test/ruby/test_dir_m17n.rb (test_filename_extutf8): use filesystem
  encoding when reading entries and comparing.

* test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_name): removed.

* test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_jp_name): split test.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36509 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-23 23:32:29 +00:00
nobu
eec0b2d88a * dir.c (dir_inspect), io.c (rb_io_inspect): keep encoding of path.
[Bug #6072]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34795 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-02-24 22:53:51 +00:00
nobu
85738261a5 * dir.c (dir_initialize): keep path in original encoding.
* error.c (syserr_initialize): prefer the encoding of message over
  locale.  [ruby-dev:45279][Bug #6071]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34794 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-02-24 22:53:42 +00:00
usa
d815916071 * test/ruby/test_process.rb (TestProcess#test_no_curdir): skip silently on
Windows, because this tests a platform specific feature and it'll never be
  supported on ruby on Windows.

* test/ruby/test_dir_m17n.rb (TestDir_M17N#test_filename_extutf8_invalid,
  TestDir_M17N#test_filename_as_bytes_extutf8): ditto.

* test/open-uri/test_open-uri.rb
  (TestOpenURI#test_find_proxy_case_sensitive_env): ditto.

* test/dl/test_handle.rb (DL::TestHandle#test_NEXT,
  DL::TestHandle#test_DEFAULT): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-03-25 02:50:17 +00:00
usa
6b15bccfdf * test/ruby/test_dir_m17n.rb: changes for windows.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27857 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-05-17 02:07:00 +00:00
muraken
31bc607cf1 * test/ruby/test_dir_m17n.rb: HFS+ escapes invalid byte sequences of filenames.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-03-25 02:37:22 +00:00
akr
fe98a03c53 * ext/socket/extconf.rb: suppress a warning.
* ext/extmk.rb: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26426 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-01-26 12:34:51 +00:00
akr
1e7f2caa1f add test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25846 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-18 13:02:28 +00:00
akr
31ee9fd966 add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25803 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-16 13:45:59 +00:00
akr
5f221c31ed add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-15 17:58:14 +00:00
akr
e88009f40e add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-15 17:14:43 +00:00
akr
453890a972 tests refined.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-15 17:11:54 +00:00
akr
43bd807c43 add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-15 16:24:45 +00:00