archive/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2025-08-16 22:19:02 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
55987 commits 29 branches 1174 tags 479 MiB
Commit graph

380 commits

Author SHA1 Message Date
usa
9b6608c78d * transcode.c (transcode_restartable0): avoid VC++6's bug. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 16:02:10 +00:00
akr
56343375b8 * include/ruby/encoding.h (rb_econv_option_t): removed. Since 
rb_econv_option_t has only one field, int flags, rb_econv_option_t is
  replaced by int.

* include/ruby/io.h: follow the above change.

* io.c: ditto.

* transcode.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 15:11:46 +00:00
akr
1afc1b7bc8 * transcode.c (rb_transcoding): moved from transcode_data.h. 
(TRANSCODING_READBUF): ditto.
  (TRANSCODING_WRITEBUF): ditto.
  (TRANSCODING_STATE_EMBED_MAX): ditto.
  (TRANSCODING_STATE): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:34:09 +00:00
akr
f6441bf61c * transcode_data.h (rb_transcoding): remove stateful field. 
add state field.
  (TRANSCODING_STATE): defined.
  (rb_transcoder): add fields: state_size, state_init_func,
  state_fini_func.
  change rb_transcoding* argument to void*.

* transcode.c (transcode_restartable0): use TRANSCODING_STATE for
  first arguments of transcoder functions.
  (rb_transcoding_open_by_transcoder): initialize state field.
  (rb_transcoding_close): finalize state field.

* tool/transcode-tblgen.rb: provide state size/init/fini.

* enc/trans/newline.trans (universal_newline_init): defined.
  (fun_so_universal_newline): take void* as a state pointer.
  (rb_universal_newline): provide state size/init/fini.
  (rb_crlf_newline): ditto.
  (rb_cr_newline): ditto.

* enc/trans/iso2022.trans (iso2022jp_init): defined.
  (fun_si_iso2022jp_to_eucjp): take void* as a state pointer.
  (fun_so_iso2022jp_to_eucjp): ditto.
  (fun_so_eucjp_to_iso2022jp): ditto.
  (iso2022jp_reset_sequence_size): ditto.
  (finish_eucjp_to_iso2022jp): ditto.
  (rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini.
  (rb_EUC_JP_to_ISO_2022_JP): ditto.

* enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a
  state pointer.
  (fun_so_to_utf_16be): ditto.
  (fun_so_from_utf_16le): ditto.
  (fun_so_to_utf_16le): ditto.
  (fun_so_from_utf_32be): ditto.
  (fun_so_to_utf_32be): ditto.
  (fun_so_from_utf_32le): ditto.
  (fun_so_to_utf_32le): ditto.
  (rb_from_UTF_16BE): provide state size/init/fini.
  (rb_to_UTF_16BE): ditto.
  (rb_from_UTF_16LE): ditto.
  (rb_to_UTF_16LE): ditto.
  (rb_from_UTF_32BE): ditto.
  (rb_to_UTF_32BE): ditto.
  (rb_from_UTF_32LE): ditto.
  (rb_to_UTF_32LE): ditto.

* enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state
  pointer.
  (fun_so_sjis2eucjp): ditto.
  (rb_eucjp2sjis): provide state size/init/fini.
  (rb_sjis2eucjp): provide state size/init/fini.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:12:06 +00:00
akr
6378201f8b * transcode_data.h (WORDINDEX_SHIFT_BITS): defined. 
(WORDINDEX2INFO): defined.
  (INFO2WORDINDEX): defined.

* tool/transcode-tblgen.rb: use WORDINDEX2INFO.

* transcode.c: use INFO2WORDINDEX.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:35:29 +00:00
akr
6fe24d1f13 refactor last change. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:09:59 +00:00
akr
069f85f812 * transcode.c (BL_MIN_BYTE): defined. 
(BL_MAX_BYTE): defined.
  (BL_OFFSET): defined.
  (BL_ACTION): defined.
  (transcode_restartable0): use BL_MIN_BYTE, BL_MAX_BYTE and
  BL_ACTION.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:08:27 +00:00
akr
ac5631e6f0 * transcode.c (BYTE_ADDR): defined. 
(WORD_ADDR): ditto.
  (BL_BASE): use BYTE_ADDR and WORD_ADDR.
  (BL_INFO): use WORD_ADDR.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 11:59:50 +00:00
akr
41e1933fd8 * transcode_data.h (base_element): removed. 
(BYTE_LOOKUP): removed.
  (BYTE_LOOKUP_BASE): don't cast.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): unsigned int, instead of uintptr_t.
  (rb_transcoding): change type of next_field, conv_tree_start and
  word_array.

* tool/transcode-tblgen.rb: generate word_array as array of unsigned
  int.

* transcode.c (transcode_restartable0): follow the above type change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 13:26:02 +00:00
akr
6058eeaf03 * tool/transcode-tblgen.rb: record infos and BYTE_LOOKUPs as index of 
word_array to avoid relocation.

* transcode.c (transcode_restartable0): add word_array to get infos
  and BYTE_LOOKUPs.

* transcode_data.h (BYTE_LOOKUP_INFO): change return type to
  uintptr_t.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 18:06:28 +00:00
akr
7908180df1 * tool/transcode-tblgen.rb: record offsets array as index of 
byte_array to avoid relocation.

* transcode.c (transcode_restartable0): add byte_array to get offsets
  array.

* transcode_data.h (BYTE_LOOKUP_BASE): change return type to
  uintptr_t.
  (rb_transcoder): add fields: byte_array, word_array and word_size.

* enc/trans/newline.trans: follow rb_transcoder change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 17:40:32 +00:00
akr
752e053a1d * transcode_data.h (BYTE_LOOKUP): change to uintptr_t array. 
(BYTE_LOOKUP_BASE): follow the type change.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): ditto.
  (rb_transcoding): ditto.

* tool/transcode-tblgen.rb: follow the type change.

* transcode.c: ditto.

* enc/trans/newline.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 16:22:49 +00:00
akr
2ae7b0328e * transcode_data.h (BYTE_LOOKUP_BASE): abstract accessor for 
BYTE_LOOKUP.
  (BYTE_LOOKUP_INFO): ditto.

* transcode.c (transcode_restartable0): use BYTE_LOOKUP_BASE and
  BYTE_LOOKUP_INFO.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 14:33:56 +00:00
akr
e7ee4fabd4 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 15:31:14 +00:00
akr
a017cecf42 fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 14:24:51 +00:00
akr
c3434e35ac rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:38:12 +00:00
akr
69610b07c3 * transcode.c (transcode.c): set source_encoding and 
destination_encoding as encoding object.
  (ecerr_source_encoding): new method.
  (ecerr_destination_encoding): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:08:31 +00:00
akr
72ee10b19c * transcode.c (make_econv_exception): rename instance variable names 
for storing encoding names.
  (ecerr_source_encoding_name): method renamed.
  (ecerr_destination_encoding_name): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 08:17:48 +00:00
akr
2d71485ec1 * transcode.c (econv_putback): associate encoding to the result. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:59:03 +00:00
akr
d427cf3af8 * transcode.c (econv_insert_output): raise ArgumentError on failure. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:43:19 +00:00
akr
c3f55e6d48 * transcode.c (econv_last_error): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:27:52 +00:00
akr
9d2accff2b * transcode.c (econv_primitive_convert): make two arguments, 
destination_byteoffset and destination_bytesize, optional.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:18:29 +00:00
akr
f3c43ae0c2 * transcode.c (make_econv_exception): error message simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:12:32 +00:00
akr
d9c5f88abc * transcode.c (econv_putback): make max argument optional. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:39:16 +00:00
akr
97a5808a14 * transcode.c (rb_econv_putback): putback from last byte. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:36:38 +00:00
akr
998a4ec5af * transcode.c (econv_putback): renamed from econv_primitive_putback. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:27:35 +00:00
akr
5bbc51ed43 * transcode.c (econv_insert_output): renamed from 
econv_primitive_insert_output.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18964 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:23:23 +00:00
akr
5e222d6945 * transcode.c (econv_primitive_errinfo): 5th element (partial_input) 
removed from result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 18:45:48 +00:00
akr
c2ba5cf9c5 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-29 16:25:27 +00:00
akr
d4afbe49f3 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:28:52 +00:00
akr
2c80188627 * transcode.c (econv_finish): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18913 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:00:02 +00:00
akr
23d10cbb00 * transcode.c (econv_convert): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:46:18 +00:00
akr
ad8cfb1b94 * transcode.c (econv_primitive_convert): accept nil as 
destination_bytesize for unlimited destination size.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18911 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:39:02 +00:00
akr
3811bb53bd * transcode.c (econv_primitive_convert): accept nil as input for empty 
input.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18910 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:13:49 +00:00
akr
c23ec90e81 * transcode.c (sym_invalid_byte_sequence): new variable. 
(sym_undefined_conversion): ditto.
  (sym_destination_buffer_full): ditto.
  (sym_source_buffer_empty): ditto.
  (sym_finished): ditto.
  (sym_output_followed_by_input): ditto.
  (sym_incomplete_input): ditto.
  (econv_result_to_symbol): use above variables.
  (Init_transcode): initialize above variables.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18908 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 16:59:17 +00:00
akr
1366802a1f * transcode.c (transcode_loop): simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18876 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 16:14:49 +00:00
akr
db6ec3105e * include/ruby/encoding.h (rb_econv_result_t): enumeration constant: 
econv_incomplete_input. 

* io.c (finish_writeconv): check econv_incomplete_input.

* transcode.c (transcode_restartable0): return econv_incomplete_input
  for unexpected end of source buffer.
  (trans_sweep): check econv_incomplete_input.
  (rb_trans_conv): ditto.
  (rb_econv_convert0): ditto.
  (rb_econv_convert): ditto.
  (transcode_loop): ditto.
  (make_econv_exception): change message for econv_incomplete_input.
  (econv_result_to_symbol): return :incomplete_input for
  econv_incomplete_input.
  (ecerr_incomplete_input): new method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 16:09:29 +00:00
akr
a68e409eaa * include/ruby/encoding.h (rb_econv_elem_t): move to transcode.c 
(rb_econv_t): defined as an incomplete type.

* transcode.c (rb_econv_elem_t): moved from encoding.h.
  (rb_econv_t): complete type defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 15:01:11 +00:00
akr
19ab08653e * transcode.c (rb_econv_open): disable newline conversion for ASCII 
incompatible encodings.
  (str_transcode0): don't need disable newline conversion here.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18870 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 12:55:14 +00:00
akr
18b097b73b * transcode.c (rb_econv_binmode): binmode is effective only once. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 12:45:20 +00:00
akr
873987f2e0 * transcode.c (str_transcode0): disable newline conversion for ASCII 
incompatible encoding.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18853 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 16:27:23 +00:00
akr
1a805010ed * transcode.c (str_transcode0): don't short cut for newline conversion. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 15:58:35 +00:00
akr
86154f63b3 * transcode.c (make_econv_exception): show readagain part for invalid 
byte sequence exception.  store the part as an instance variable.
  (ecerr_readagain_bytes): new method to access the readagain part.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18850 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 15:26:54 +00:00
akr
0486026389 * transcode.c (rb_econv_open): make last_tc NULL if there are only 
additional transcoders.
  (econv_description): extracted from rb_econv_open_exc.
  (rb_econv_open_exc): use econv_description.
  (econv_inspect): use econv_description.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18843 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 15:01:36 +00:00
akr
aeffc67dbb * transcode.c (rb_econv_open_by_transcoder_entries): initialize 
last_error.  num_trans may be zero.
  (rb_econv_convert0): num_trans may be zero.
  (rb_econv_putbackable): ditto.
  (rb_econv_putback): ditto.
  (rb_econv_convert): input_ptr and output_ptr may be NULL.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 13:04:16 +00:00
nobu
10906176b4 * transcode.c (rb_econv_substr_append): fix for buffer overrun. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 08:35:42 +00:00
nobu
8c6040edca * transcode.c (rb_econv_open_exc): constified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18828 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 01:42:36 +00:00
akr
2c78899641 * configure.in (BUILTIN_TRANSSRCS): defined. 
(BUILTIN_TRANSOBJS): defined.

* enc/Makefile.in (BUILTIN_TRANSES): defined.

* enc/make_encmake.rb (BUILTIN_TRANSES): defined.

* enc/depend: don't generate rules for builtin transcoders.

* common.mk (COMMONOBJS): add BUILTIN_TRANSOBJS.
  (enc.mk): pass BUILTIN_TRANSOBJS.
  (newline.c): new rule.
  (newline.$(OBJEXT)): new ru.e
  (srcs): newline.c added.

* Makefile.in (BUILTIN_TRANSSRCS): defined.
  (BUILTIN_TRANSOBJS): defined.

* transcode.c (Init_transcode): call Init_newline.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18826 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 15:58:43 +00:00
akr
bf5e559ce0 * transcode.c (declare_transcoder): arguments order changed. 
(rb_declare_transcoder): call declare_transcoder just once.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18824 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 15:13:37 +00:00
akr
ae1b02a8da * include/ruby/encoding.h (rb_econv_t): use rb_econv_option_t. 
* transcode.c: follow the rb_econv_t change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 10:49:36 +00:00