Commit graph

1411 commits

Author SHA1 Message Date
ngoto
a92a537bf4 * string.c: Specify termlen as far as possible.
Additional fix for [Bug #12536] [ruby-dev:49699].

* string.c (rb_usascii_str_new, rb_utf8_str_new): Specify termlen
  which is apparently 1 for the encodings.

* string.c (str_new0_cstr): New static function to create a String
  object from a C string with specifying termlen.

* string.c (rb_usascii_str_new_cstr, rb_utf8_str_new_cstr): Specify
  termlen by using new str_new0_cstr().

* string.c (str_new_static): Specify termlen from the given encoding
  when creating a new String object is needed.

* string.c (rb_tainted_str_new_with_enc): New function to create a
  tainted String object with the given encoding. This means that
  the termlen is correctly specified. Curretly static function.
  The function name might be renamed to rb_tainted_enc_str_new
  or rb_enc_tainted_str_new.

* string.c (rb_external_str_new_with_enc): Use encoding by using the
  above rb_tainted_str_new_with_enc().


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55555 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-01 11:24:11 +00:00
ngoto
10e28726a1 * string.c (rb_str_subseq, str_substr): When RSTRING_EMBED_LEN_MAX
is used, TERM_LEN(str) should be considered with it because
  embedded strings are also processed by TERM_FILL.
  Additional fix for [Bug #12536] [ruby-dev:49699].


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55552 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-01 04:50:38 +00:00
ngoto
6734a0c3d9 string.c: Add parentheses to avoid C source code ambiguity. [Bug #12536]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-01 03:58:51 +00:00
ngoto
f2ee22371b * string.c: Fix memory corruptions when using UTF-16/32 strings.
[Bug #12536] [ruby-dev:49699]

* string.c (TERM_LEN_MAX): Macro for the longest TERM_FILL length,
  the same as largest value of rb_enc_mbminlen(enc) among encodings.

* string.c (str_new, rb_str_buf_new, str_shared_replace): Allocate
  +TERM_LEN_MAX bytes instead of +1. This change may increase memory
  usage.

* string.c (rb_str_new_with_class): Use TERM_LEN of the "obj".

* string.c (rb_str_plus, rb_str_justify): Use str_new0 which is aware
  of termlen.

* string.c (str_shared_replace): Copy +termlen bytes instead of +1.

* string.c (rb_str_times): termlen should not be included in capa.

* string.c (RESIZE_CAPA_TERM): When using RSTRING_EMBED_LEN_MAX,
  termlen should be counted with it because embedded strings are
  also processed by TERM_FILL.

* string.c (rb_str_capacity, str_shared_replace, str_buf_cat): ditto.

* string.c (rb_str_drop_bytes, rb_str_setbyte, str_byte_substr): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55547 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-30 10:20:23 +00:00
nobu
bcf0a198f1 CASEMAP_DEBUG [ci skip]
* string.c (rb_str_casemap, rb_str_ascii_casemap): move
  debug/tuning messages under a preprocessor condition,
  CASEMAP_DEBUG.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55483 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-21 08:19:59 +00:00
nobu
3a6bb56029 Fix garbage allocation
* string.c (rb_str_casemap): do not put code with side effects
  inside RSTRING_PTR() macro which evaluates the argument multiple
  times.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-21 07:38:16 +00:00
naruse
8272729977 * string.c (rb_str_casemap): fix memory leak.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55480 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-21 07:14:05 +00:00
naruse
9d291c82e5 * string.c (rb_str_casemap): int is too small for string size.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-21 07:14:04 +00:00
nobu
1cbc622ea7 string.c: adjust buffer size
* string.c (tr_trans): adjust buffer size by processed and rest
  lengths, instead of doubling repeatedly.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55428 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-16 03:17:54 +00:00
nobu
cc9f1e9195 string.c: fix terminator
* string.c (tr_trans): consider terminator length and fix heap
  overflow.  reported by Guido Vranken <guido AT guidovranken.nl>.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55427 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-16 02:15:27 +00:00
nobu
aaf8c09900 Fix typo in string.c [ci skip]
* string.c (rb_str_oct): [DOC] fix typo, hornored -> honored.
  [Fix GH-1379]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55378 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-11 06:02:46 +00:00
duerst
02f7ad6237 * enc/iso_8859_1.c: Implement non-ASCII case mapping.
* test/ruby/enc/test_case_comprehensive.rb: Tests for above.
* string.c: Add iso-8859-1 to supported encodings.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-11 00:46:21 +00:00
duerst
10174c295b * string.c: Special-case :ascii option in rb_str_capitalize_bang and
rb_str_swapcase_bang.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55361 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-10 08:35:17 +00:00
duerst
13f576d6b9 * string.c: Special-case :ascii option in rb_str_upcase_bang (retry).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-10 08:12:28 +00:00
nobu
2667d1b38f hash.c: ensure NUL-terminated for ENV
* hash.c (get_env_cstr): ensure NUL-terminated.
  [ruby-dev:49655] [Bug #12475]
* string.c (rb_str_fill_terminator): return the pointer to the
  NUL-terminated content.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55345 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-10 05:48:38 +00:00
kazu
075cf3d2e8 string.c (rb_str_ascii_casemap): fix compile error.
error: implicit conversion loses integer precision: 'long' to 'int' [-Werror,-Wshorten-64-to-32]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55332 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-08 14:11:17 +00:00
duerst
872f9a498f * string.c: Revert previous commit (possibility of endless loop).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55331 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-08 13:22:28 +00:00
duerst
5eb73eeda8 * string.c: Special-case :ascii option in rb_str_upcase_bang.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55330 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-08 12:57:44 +00:00
duerst
f0fc6ec872 * string.c: New static function rb_str_ascii_casemap; special-casing
:ascii option in rb_str_upcase_bang and rb_str_downcase_bang.
* regenc.c: Fix a bug (wrong use of unnecessary slack at end of string).
* regenc.h -> include/ruby/oniguruma.h: Move declaration of
  onigenc_ascii_only_case_map so that it is visible in string.c.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55329 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-08 12:28:42 +00:00
duerst
8743f010c6 * string.c (rb_str_upcase_bang, rb_str_capitalize_bang,
rb_str_swapcase_bang): Switch to use primitive.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-07 08:18:42 +00:00
duerst
53a3e3ddd9 * string.c (rb_str_downcase_bang): Switch to use primitive except if
conversion can be done ASCII-only.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-07 07:44:19 +00:00
duerst
ab5f23f26c * string.c: Added UTF-16BE/LE and UTF-32BE/LE to supported encodings
for Unicode case mapping.
* test/ruby/enc/test_case_comprehensive.rb: Tests for above
  functionality; fixed an encoding issue in assertion error message.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55296 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-06 09:36:36 +00:00
duerst
2f49aa8f62 * string.c Change rb_str_casemap to use encoding primitive
case_map instead of directly calling onigenc_unicode_case_map.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55293 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-06 04:37:10 +00:00
duerst
c5ea268264 * string.c: Remove :lithuanian guard for Unicode case mapping.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-05 05:46:37 +00:00
nobu
40c3c3ec6c crypt.h: remove initialized
* missing/crypt.h (struct crypt_data): remove unnecessary member
  "initialized".
* missing/crypt.c (des_setkey_r): nothing to be initialized in
  crypt_data.
* configure.in (struct crypt_data): check for "initialized" in
  struct crypt_data, which may be only in glibc, and isn't on AIX
  at least.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55272 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-04 01:54:54 +00:00
duerst
3dd98b2446 * string.c: Raise ArgumentError when invalid string is detected in
case mapping methods.
* enc/unicode.c: Check for invalid string and signal with negative
  length value.
* test/ruby/enc/test_case_mapping.rb: Add tests for above.
* test/ruby/test_m17n_comb.rb: Add a message to clarify test failure.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-02 01:24:52 +00:00
nobu
a94201243e string.c: fallback to crypt_r
* string.c: prefer crypt_r to crypt iff system crypt nor crypt_r
  are not provided.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-01 13:17:31 +00:00
nobu
a8bfa9bdf1 use system crypt
* configure.in: revert r55237.  replace crypt, not crypt_r, and
  check if crypt is broken more.
* missing/crypt.c: move crypt_r.c
* string.c (rb_str_crypt): use crypt_r if provided by the system.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55245 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-01 06:58:21 +00:00
nobu
3c31685e11 use crypt_r
* string.c (rb_str_crypt): use reentrant crypt_r.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55237 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-01 00:48:08 +00:00
naruse
e6ff652ce8 Revert r55225
Run test-all before large commit:
"* string.c: Activate full Unicode case mapping for UTF-8 by removing"

This reverts commit 3fb0fcd1e8.
20160531T013303Z.fail.html.gz

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55226 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-31 02:56:09 +00:00
duerst
3fb0fcd1e8 * string.c: Activate full Unicode case mapping for UTF-8 by removing
the protective check for the presence of an option.
  Update documentation.
* test/ruby/enc/test_case_comprehensive.rb: Adjust tests for above change.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-31 01:10:06 +00:00
duerst
ae4fba3167 * string.c: Document current behavior for other case mapping methods
on String. [ci skip]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55217 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-30 12:15:41 +00:00
duerst
85950c5257 * string.c: Document current situation for String#downcase. [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55215 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-30 11:00:26 +00:00
nobu
79a85b18cc string.c: return reallocated pointer
* string.c (str_fill_term): return new pointer reallocated by
  filling terminator.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55212 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-30 07:20:28 +00:00
nobu
9ac5f9135a string.c: get rid of unnecessary empty string
* string.c (str_substr, rb_str_aref): refactor not to create
  unnecessary empty string.
* string.c (str_byte_substr, str_byte_aref): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55209 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-30 05:50:27 +00:00
nobu
e3e8cae9be string.c: check in the order
* string.c (rb_str_aref_m, rb_str_byteslice): check arguments in
  the left-to-right order.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55208 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-30 05:41:02 +00:00
nobu
4fad63da01 transcode.c: scrub in the given encoding
* transcode.c (str_transcode0): scrub in the given encoding when
  the source encoding is given, not in the encoding of the
  receiver.  [ruby-core:75732] [Bug #12431]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55181 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-27 08:09:46 +00:00
nobu
b493d156de string.c: integer overflow
* string.c (rb_str_modify_expand): check integer overflow.
  [ruby-core:75592] [Bug #12390]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55054 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-18 05:52:40 +00:00
nobu
4a9705d6e3 ruby.h: RB_INTEGER_TYPE_P
* include/ruby/ruby.h (RB_INTEGER_TYPE_P): new macro and
  underlying inline function to check if the object is an
  Integer (Fixnum or Bignum).

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-18 01:17:43 +00:00
naruse
28f5e12c24 * configure.in: check function attirbute const and pure,
and define CONSTFUNC and PUREFUNC if available.
  Note that I don't add those options as default because
  it still shows many false-positive (it seems not to consider
  longjmp).

* vm_eval.c (stack_check): get rb_thread_t* as an argument
  to avoid duplicate call of GET_THREAD().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54952 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-08 17:44:51 +00:00
yui-knk
deca1d8007 * string.c (rb_str_sub): Fix a special match variable name.
[ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-05 05:39:35 +00:00
naruse
cdef0bc833 * string.c (count_utf8_lead_bytes_with_word): Use __builtin_popcount
only if it can use SSE 4.2 POPCNT whose latency is 3 cycle.

* internal.h (rb_popcount64): use __builtin_popcountll because now
  it is in fast path.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54894 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-03 13:14:30 +00:00
nobu
c353ec0c9e string.c: shortcut
* string.c (rb_str_concat): shortcut concatenation to ASCII-8BIT
  as well as US-ASCII.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54882 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-02 03:58:28 +00:00
nobu
321c6df89b string.c: fix doc
* string.c (rb_str_concat): [DOC] fix the indefinite article, for
  replacement from Fixnum to Integer.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-02 03:53:34 +00:00
nobu
0e3475a6d9 string.c: fix braces
* string.c (search_nonascii): fix braces unmatched by a
  preprocessing condition.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54879 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-02 00:06:04 +00:00
naruse
2fc973796a fix mixed declaration on non UNALIGNED_WORD_ACCESS
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54877 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-01 18:27:41 +00:00
naruse
64837f778a fix for where UNALIGNED_WORD_ACCESS is not allowed
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54867 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-01 14:19:02 +00:00
naruse
db2c32778d Use WORDS_BIGENDIAN
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-01 09:07:14 +00:00
naruse
0f0121fe1a * string.c (search_nonascii): use nlz on big endian environments.
* internal.h (nlz_intpr): defined.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54859 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-04-30 22:32:05 +00:00
naruse
424a706afe More optimization for r54854's search_nonascii
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54857 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-04-30 16:32:36 +00:00