archive/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2025-08-15 21:49:06 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
65041 commits 29 branches 1174 tags 476 MiB
Commit graph

398 commits

Author SHA1 Message Date
akr
dc98e2528a * transcode.c (output_hex_charref): upcase hexadecimal digits. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19163 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 20:37:36 +00:00
akr
c500c37fbc * include/ruby/encoding.h (ECONV_UNDEF_HEX_CHARREF): defined. 
* transcode.c (output_hex_charref): new function.
  (rb_econv_convert): call output_hex_charref if
  ECONV_UNDEF_HEX_CHARREF.
  (Init_transcode): Encoding::Converter::UNDEF_HEX_CHARREF added.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 20:24:18 +00:00
akr
7d3598ee60 * transcode.c (rb_econv_convert): use ECONV_INVALID_MASK and 
ECONV_UNDEF_MASK.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19161 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 19:03:21 +00:00
akr
80abde2c89 * transcode.c (allocate_converted_string): add arguments for a buffer 
allocated by caller.
  (rb_econv_insert_output): provide caller allocated buffer to
  allocate_converted_string.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19159 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 18:20:20 +00:00
akr
f0d9f949e8 * transcode.c (str_transcode_enc_args): local variables renamed. 
(str_transcode0): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19158 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:59:32 +00:00
akr
e93289c03f update rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:25:56 +00:00
akr
134ca08dd1 * transcode.c (econv_s_stateless_encoding): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:23:49 +00:00
akr
f2fd6d3b15 * transcode.c (enc_arg): extracted from str_transcode_enc_args. 
(str_transcode_enc_args): use enc_arg.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19154 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:02:33 +00:00
akr
f549f1ae32 * transcode_data.h (rb_transcoder): rename fields: 
from_encoding -> src_encoding, to_encoding -> dst_encoding.

* transcode.c: follow the renaming.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 15:46:18 +00:00
akr
a50b307900 * transcode.c: variables renamed for consistency. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 15:35:30 +00:00
akr
0406b0f154 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 13:17:40 +00:00
akr
5da4e660ad * transcode.c (econv_init): accept an integer as 3rd argument as well. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19132 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 12:55:02 +00:00
akr
2494e67f16 * transcode.c (sym_partial_input): new variable. 
(econv_primitive_convert): accept a hash as 5th argument as well.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 12:48:21 +00:00
akr
f7607eff9a * transcode.c (sym_universal_newline_decoder): new variable. 
(sym_crlf_newline_encoder): ditto.
  (sym_cr_newline_encoder): ditto.
  (econv_opts): check newline converter options.
  (econv_init): make 3rd argument hash/nil only.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 12:08:54 +00:00
akr
0031967d20 * transcode_data.h (PType): defined unconditionaly. 
* transcode.c (PType): don't define here.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 11:31:37 +00:00
akr
45c3707303 * include/ruby/encoding.h (ECONV_INVALID_IGNORE): removed because 
it tend to cause security problem.  If the behaviour is really
  required, ECONV_INVALID_REPLACE with empty string can be used.
  For example, CVE-2006-2313, CVE-2008-1036, [ruby-core:15645]
  (ECONV_UNDEF_IGNORE): ditto.

* transcode.c (rb_econv_convert): follow the above change.
  (econv_opts): ditto.
  (Init_transcode): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 10:15:34 +00:00
akr
58964e3249 * include/ruby/io.h (rb_io_t): new fields: encs.ecopts and 
writeconv_pre_ecopts.
  (MakeOpenFile): initialize them.

* include/ruby/encoding.h (rb_str_transcode): take ecopts argument.
  (rb_econv_flags): removed.
  (rb_econv_prepare_opts): declared.
  (rb_econv_open_opts): declared.

* io.c (make_writeconv): use rb_econv_open_opts.
  (make_readconv): ditto.
  (io_fwrite): follow rb_str_transcode change.
  (rb_io_extract_modeenc): use rb_econv_prepare_opts.
  (rb_file_open_generic): initialize encs.ecopts.
  (rb_file_open_internal): ditto.
  (rb_io_reopen): ditto.
  (argf_ecopts): defined.
  (argf_next_argv): set encs.ecopts.
  (io_encoding_set): use rb_econv_prepare_opts.
  (argf_set_encoding): set argf_ecopts.

* gc.c (gc_mark_children): mark encs.ecopts and writeconv_pre_ecopts
  in T_FILE.

* transcode.c (transcode_loop): take ecopts argument. use
  rb_econv_open_opts.
  (rb_econv_flags): removed.
  (rb_econv_prepare_opts): defined.
  (rb_econv_open_opts): defined.
  (str_transcode0): take ecopts.
  (str_transcode): use rb_econv_prepare_opts.
  (rb_str_transcode): take ecopts.
  (econv_init): accept hash argument.
  (econv_insert_output): follow rb_str_transcode change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 18:18:10 +00:00
akr
0ebcad6a7b * include/ruby/encoding.h (rb_econv_set_replacemenet): declared. 
* transcode.c (rb_econv_t): new fields: replacement_str,
  replacement_len, replacement_enc and replacement_allocated.
  (get_replacement_character): make len as size_t.
  (rb_econv_open_by_transcoder_entries): initialize the new fields.
  (rb_econv_close): deallocate replacement_str if it allocated.
  (make_replacement): new function.
  (output_replacement_character): use make_replacement.
  (rb_econv_set_replacemenet): defined.
  (econv_get_replacement): new method.
  (econv_set_replacement): new method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19108 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 16:34:11 +00:00
usa
9b6608c78d * transcode.c (transcode_restartable0): avoid VC++6's bug. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 16:02:10 +00:00
akr
56343375b8 * include/ruby/encoding.h (rb_econv_option_t): removed. Since 
rb_econv_option_t has only one field, int flags, rb_econv_option_t is
  replaced by int.

* include/ruby/io.h: follow the above change.

* io.c: ditto.

* transcode.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 15:11:46 +00:00
akr
1afc1b7bc8 * transcode.c (rb_transcoding): moved from transcode_data.h. 
(TRANSCODING_READBUF): ditto.
  (TRANSCODING_WRITEBUF): ditto.
  (TRANSCODING_STATE_EMBED_MAX): ditto.
  (TRANSCODING_STATE): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:34:09 +00:00
akr
f6441bf61c * transcode_data.h (rb_transcoding): remove stateful field. 
add state field.
  (TRANSCODING_STATE): defined.
  (rb_transcoder): add fields: state_size, state_init_func,
  state_fini_func.
  change rb_transcoding* argument to void*.

* transcode.c (transcode_restartable0): use TRANSCODING_STATE for
  first arguments of transcoder functions.
  (rb_transcoding_open_by_transcoder): initialize state field.
  (rb_transcoding_close): finalize state field.

* tool/transcode-tblgen.rb: provide state size/init/fini.

* enc/trans/newline.trans (universal_newline_init): defined.
  (fun_so_universal_newline): take void* as a state pointer.
  (rb_universal_newline): provide state size/init/fini.
  (rb_crlf_newline): ditto.
  (rb_cr_newline): ditto.

* enc/trans/iso2022.trans (iso2022jp_init): defined.
  (fun_si_iso2022jp_to_eucjp): take void* as a state pointer.
  (fun_so_iso2022jp_to_eucjp): ditto.
  (fun_so_eucjp_to_iso2022jp): ditto.
  (iso2022jp_reset_sequence_size): ditto.
  (finish_eucjp_to_iso2022jp): ditto.
  (rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini.
  (rb_EUC_JP_to_ISO_2022_JP): ditto.

* enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a
  state pointer.
  (fun_so_to_utf_16be): ditto.
  (fun_so_from_utf_16le): ditto.
  (fun_so_to_utf_16le): ditto.
  (fun_so_from_utf_32be): ditto.
  (fun_so_to_utf_32be): ditto.
  (fun_so_from_utf_32le): ditto.
  (fun_so_to_utf_32le): ditto.
  (rb_from_UTF_16BE): provide state size/init/fini.
  (rb_to_UTF_16BE): ditto.
  (rb_from_UTF_16LE): ditto.
  (rb_to_UTF_16LE): ditto.
  (rb_from_UTF_32BE): ditto.
  (rb_to_UTF_32BE): ditto.
  (rb_from_UTF_32LE): ditto.
  (rb_to_UTF_32LE): ditto.

* enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state
  pointer.
  (fun_so_sjis2eucjp): ditto.
  (rb_eucjp2sjis): provide state size/init/fini.
  (rb_sjis2eucjp): provide state size/init/fini.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:12:06 +00:00
akr
6378201f8b * transcode_data.h (WORDINDEX_SHIFT_BITS): defined. 
(WORDINDEX2INFO): defined.
  (INFO2WORDINDEX): defined.

* tool/transcode-tblgen.rb: use WORDINDEX2INFO.

* transcode.c: use INFO2WORDINDEX.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:35:29 +00:00
akr
6fe24d1f13 refactor last change. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:09:59 +00:00
akr
069f85f812 * transcode.c (BL_MIN_BYTE): defined. 
(BL_MAX_BYTE): defined.
  (BL_OFFSET): defined.
  (BL_ACTION): defined.
  (transcode_restartable0): use BL_MIN_BYTE, BL_MAX_BYTE and
  BL_ACTION.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:08:27 +00:00
akr
ac5631e6f0 * transcode.c (BYTE_ADDR): defined. 
(WORD_ADDR): ditto.
  (BL_BASE): use BYTE_ADDR and WORD_ADDR.
  (BL_INFO): use WORD_ADDR.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 11:59:50 +00:00
akr
41e1933fd8 * transcode_data.h (base_element): removed. 
(BYTE_LOOKUP): removed.
  (BYTE_LOOKUP_BASE): don't cast.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): unsigned int, instead of uintptr_t.
  (rb_transcoding): change type of next_field, conv_tree_start and
  word_array.

* tool/transcode-tblgen.rb: generate word_array as array of unsigned
  int.

* transcode.c (transcode_restartable0): follow the above type change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 13:26:02 +00:00
akr
6058eeaf03 * tool/transcode-tblgen.rb: record infos and BYTE_LOOKUPs as index of 
word_array to avoid relocation.

* transcode.c (transcode_restartable0): add word_array to get infos
  and BYTE_LOOKUPs.

* transcode_data.h (BYTE_LOOKUP_INFO): change return type to
  uintptr_t.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 18:06:28 +00:00
akr
7908180df1 * tool/transcode-tblgen.rb: record offsets array as index of 
byte_array to avoid relocation.

* transcode.c (transcode_restartable0): add byte_array to get offsets
  array.

* transcode_data.h (BYTE_LOOKUP_BASE): change return type to
  uintptr_t.
  (rb_transcoder): add fields: byte_array, word_array and word_size.

* enc/trans/newline.trans: follow rb_transcoder change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 17:40:32 +00:00
akr
752e053a1d * transcode_data.h (BYTE_LOOKUP): change to uintptr_t array. 
(BYTE_LOOKUP_BASE): follow the type change.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): ditto.
  (rb_transcoding): ditto.

* tool/transcode-tblgen.rb: follow the type change.

* transcode.c: ditto.

* enc/trans/newline.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 16:22:49 +00:00
akr
2ae7b0328e * transcode_data.h (BYTE_LOOKUP_BASE): abstract accessor for 
BYTE_LOOKUP.
  (BYTE_LOOKUP_INFO): ditto.

* transcode.c (transcode_restartable0): use BYTE_LOOKUP_BASE and
  BYTE_LOOKUP_INFO.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 14:33:56 +00:00
akr
e7ee4fabd4 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 15:31:14 +00:00
akr
a017cecf42 fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 14:24:51 +00:00
akr
c3434e35ac rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:38:12 +00:00
akr
69610b07c3 * transcode.c (transcode.c): set source_encoding and 
destination_encoding as encoding object.
  (ecerr_source_encoding): new method.
  (ecerr_destination_encoding): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:08:31 +00:00
akr
72ee10b19c * transcode.c (make_econv_exception): rename instance variable names 
for storing encoding names.
  (ecerr_source_encoding_name): method renamed.
  (ecerr_destination_encoding_name): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 08:17:48 +00:00
akr
2d71485ec1 * transcode.c (econv_putback): associate encoding to the result. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:59:03 +00:00
akr
d427cf3af8 * transcode.c (econv_insert_output): raise ArgumentError on failure. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:43:19 +00:00
akr
c3f55e6d48 * transcode.c (econv_last_error): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:27:52 +00:00
akr
9d2accff2b * transcode.c (econv_primitive_convert): make two arguments, 
destination_byteoffset and destination_bytesize, optional.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:18:29 +00:00
akr
f3c43ae0c2 * transcode.c (make_econv_exception): error message simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:12:32 +00:00
akr
d9c5f88abc * transcode.c (econv_putback): make max argument optional. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:39:16 +00:00
akr
97a5808a14 * transcode.c (rb_econv_putback): putback from last byte. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:36:38 +00:00
akr
998a4ec5af * transcode.c (econv_putback): renamed from econv_primitive_putback. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:27:35 +00:00
akr
5bbc51ed43 * transcode.c (econv_insert_output): renamed from 
econv_primitive_insert_output.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18964 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:23:23 +00:00
akr
5e222d6945 * transcode.c (econv_primitive_errinfo): 5th element (partial_input) 
removed from result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 18:45:48 +00:00
akr
c2ba5cf9c5 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-29 16:25:27 +00:00
akr
d4afbe49f3 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:28:52 +00:00
akr
2c80188627 * transcode.c (econv_finish): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18913 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:00:02 +00:00
akr
23d10cbb00 * transcode.c (econv_convert): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:46:18 +00:00