Commit graph

15465 commits

Author SHA1 Message Date
akr
128abaede6 * transcode.c (econv_primitive_putback): new method.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18671 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-17 04:40:59 +00:00
akr
0051d4a077 * include/ruby/encoding.h (rb_econv_putbackable): declared.
(rb_econv_putback): ditto.

* transcode.c (rb_econv_putbackable): implemented.
  (rb_econv_putback): ditto.

* io.c (io_getc): put back bytes if possible.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18670 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-17 04:25:56 +00:00
akr
04bbc1df12 * transcode.c (make_econv_exception): add several instance variables
to exception object.
  (ecerr_source_encoding): new method:
  Encoding::ConversionUndefined#source_encoding and 
  Encoding::InvalidByteSequence#source_encoding.
  (ecerr_destination_encoding): new method:
  Encoding::ConversionUndefined#destination_encoding and 
  Encoding::InvalidByteSequence#destination_encoding.
  (econverr_error_char): new method:
  Encoding::ConversionUndefined#error_char.
  (econverr_error_bytes): new method:
  Encoding::ConversionUndefined#error_bytes.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18669 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-17 03:05:40 +00:00
nobu
21b00e0222 * random.c (struct MT): packed Mersenne Twister staffs.
* random.c (struct RandSeed): packed random seed staffs.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18668 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-17 02:43:20 +00:00
naruse
067f704f64 * test/iconv/test_option.rb (test_ignore_option): skip if iconv
doesn't have transliterate.

* test/iconv/test_option.rb (test_translit_option): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18667 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 23:40:46 +00:00
akr
63daa7c07d * include/ruby/io.h (rb_io_t): new fields: readconv, crbuf, crbuf_off,
crbuf_len, crbuf_capa.
  (MakeOpenFile): initialize them.

* io.c (io_shift_crbuf): new function.
  (io_getc): use econv.
  (rb_io_fptr_finalize): finalize readconv and crbuf.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18666 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 17:06:35 +00:00
svn
69f22784f2 * 2008-08-17
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 15:04:43 +00:00
akr
6acb0e79c8 * include/ruby/encoding.h (rb_econv_check_error): declared.
* transcode.c (make_econv_exception): new function.
  (transcode_loop): use make_econv_exception.
  (rb_econv_check_error): defined.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 15:04:39 +00:00
nobu
a4df037ffd * ChangeLog: stripped trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18657 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 09:36:34 +00:00
nobu
8b2093d3b2 * id.c (Init_id): commit miss.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18656 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 06:43:37 +00:00
akr
4dff8b2428 * include/ruby/encoding.h (rb_econv_elem_t): fields removed: from and
to.
  (rb_econv_t): new fields: source_encoding_name and
  destination_encoding_name.

* transcode.c (rb_econv_open_by_transcoder_entries): initialize the
  new fields.
  (rb_econv_open): set up the new fields.
  (econv_inspect): use the new fields.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18655 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 06:25:38 +00:00
akr
7a0bea4fd3 * include/ruby/encoding.h (rb_econv_t): add fields: in_buf_start,
in_data_start, in_data_end, in_buf_end and last_trans_index.
  (rb_econv_output): removed.
  (rb_econv_insert_output): declared.
  (rb_econv_encoding_to_insert_output): declared.

* enc/trans/newline.trans (rb_universal_newline): stateful_type
  changed.

* transcode.c (transcode_restartable0): initialize inchar_start,
  tc->recognized_len and next_table at beginning of the loop.
  (rb_econv_open_by_transcoder_entries): initialize new fields.
  (rb_econv_open): setup last_trans_index.
  (trans_sweep): last out_buf_start can be non-NULL now.
  (rb_econv_convert): check last out_buf_start and in_buf_start at
  first.
  (rb_econv_output_with_destination_encoding): removed.
  (econv_just_convert): removed.
  (rb_econv_output): removed.
  (econv_primitive_output): method removed.
  (rb_econv_encoding_to_insert_output): new function.
  (allocate_converted_string): new function.
  (rb_econv_insert_output): new function.
  (econv_primitive_insert_output): new method.
  (output_replacement_character): use rb_econv_insert_output.  unused
  arguments removed.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 05:32:42 +00:00
nobu
5f9c188d97 * include/ruby/ruby.h (rb_intern_const): tiny optimization.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18653 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 00:20:31 +00:00
nobu
c65dc8c3b1 * include/ruby/ruby.h: commit miss.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18652 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 00:07:46 +00:00
nobu
38280af483 * include/ruby/ruby.h (SSIZET2NUM, NUM2SSIZET, SSIZE_MAX, SSIZE_MIN):
macros for ssize_t.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 23:53:07 +00:00
akr
a55e167b68 * transcode_data.h (rb_transcoder_stateful_type_t): defined.
(rb_transcoder): add field: stateful_type.

* tool/transcode-tblgen.rb: generate stateful_type field as
  stateless_converter.

* enc/trans/iso2022.trans: follow rb_transcoder change.

* enc/trans/newline.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 23:13:01 +00:00
svn
aafb3af52b * 2008-08-16
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18649 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 16:41:19 +00:00
akr
c3d0206e04 add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18648 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 16:41:16 +00:00
akr
3a427c6199 * transcode.c (econv_just_convert): extracted from rb_econv_output.
(rb_econv_output): use econv_just_convert.
  (econv_primitive_output): new method.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 14:17:11 +00:00
akr
fe091e89b7 * include/ruby/encoding.h (rb_econv_output): add str_encoding
argument.

* transcode.c (get_replacement_character): add repl_enc_ptr argument.
  (rb_econv_output_with_destination_encoding): renamed from
  rb_econv_output and make it static.
  (rb_econv_output): convert str and call
  rb_econv_output_with_destination_encoding.
  (output_replacement_character): follow above interface change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18646 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 11:02:07 +00:00
nobu
a98587c0fd * string.c (rb_str_drop_bytes): use memmove.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18645 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 10:34:06 +00:00
akr
7f0c5beb6f * include/ruby/encoding.h (rb_econv_t): add error_tc in last_error.
* transcode.c (rb_econv_convert): fill error_tc.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 09:33:59 +00:00
akr
131e194cec * include/ruby/encoding.h (rb_econv_t): new field: last_error.
* transcode.c (rb_trans_conv): new argument: result_position_ptr.
  (rb_econv_convert): fill last_error.
  (econv_result_to_symbol): extracted from econv_primitive_convert.
  (econv_primitive_errinfo): new method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 09:12:56 +00:00
nobu
d6d2eed78e * common.mk (parse.{c,h}): creates in the dependency order.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18642 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 08:39:14 +00:00
nobu
fa9a533d0b * test/ruby/test_settracefunc.rb (test_call, test_class):
RubyVM::FrozenCore is no longer visible.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18641 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 08:01:17 +00:00
nobu
1be7c799e6 * vm_core.h (exec_event_hooks): skips RubyVM::FrozenCore.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 07:59:33 +00:00
nobu
3f3a0fcc8e * vm.c (Init_VM): get rid of SEGV in a trace proc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 06:30:43 +00:00
akr
e4e2ef3975 add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 02:32:47 +00:00
akr
96ae1f008f * transcode.c (conv_init): check empty name.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18637 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:34:16 +00:00
akr
8b04d998ae add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18636 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:27:03 +00:00
akr
b9d5d6a137 add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18635 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:18:42 +00:00
akr
520cd6d186 * transcode.c (econv_init): don't create dummy encoding if
rb_econv_open is failed.
  (make_dummy_encoding): new function extracted from make_encoding.
  (make_encoding): removed.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18634 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:05:06 +00:00
akr
51c7947bde ignore uncommon.mk.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18633 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 23:40:02 +00:00
akr
95af70c5e7 udpate rdoc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18632 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 16:10:57 +00:00
naruse
f9ea6ed5a9 * common.mk ({$(srcdir)}.y.c): escape backslash.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 16:09:00 +00:00
akr
8eda80ff2c * transcode.c (econv_primitive_convert): set destination_buffer
encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18630 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 16:06:33 +00:00
svn
c9453ecc04 * 2008-08-15
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18629 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 15:56:44 +00:00
akr
74a36d5d1f * include/ruby/encoding.h (rb_econv_output): declared.
* transcode_data.h (rb_transcoder): add resetsize_func field.

* enc/trans/iso2022.trans (iso2022jp_reset_sequence_size): defined.
  (rb_EUC_JP_to_ISO_2022_JP): provede resetsize_func.

* tool/transcode-tblgen.rb: set NULL for resetsize_func.

* transcode.c (rb_econv_output): new function for inserting output.
  (output_replacement_character): use rb_econv_output.
  (transcode_loop): check return value of
  output_replacement_character.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 15:56:39 +00:00
akr
e0e39e0db8 * include/ruby/encoding.h (ECONV_UNIVERSAL_NEWLINE_DECODER): defined.
(ECONV_CRLF_NEWLINE_ENCODER): ditto.
  (ECONV_CR_NEWLINE_ENCODER): ditto.
  (ECONV_PARTIAL_INPUT): ditto.
  (ECONV_OUTPUT_FOLLOWED_BY_INPUT): ditto.

* transcode.c: use ECONV_* defined as above.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18627 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 14:48:41 +00:00
akr
ed7d1a25cb * transcode.c: local variable renamed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18626 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 14:35:55 +00:00
akr
c82aee31b4 * include/ruby/encoding.h (rb_econv_result_t): moved from
transcode_data.h.
  (rb_econv_elem_t): ditto.
  (rb_econv_t): ditto.  source_encoding and destination_encoding field
  is added.
  (rb_econv_open): declared.
  (rb_econv_convert): ditto.
  (rb_econv_close): ditto.

* transcode.c (rb_econv_open_by_transcoder_entries): initialize
  source_encoding and destination_encoding field as NULL.
  (rb_econv_open): make it external linkage.
  (rb_econv_close): ditto.
  (rb_econv_convert): ditto.  renamed from rb_econv_conv.
  (make_encoding): new function.
  (econv_init): use make_encoding and store rb_encoding* in
  rb_econv_t.
  (econv_source_encoding): new method
  Encoding::Converter#source_encoding.
  (econv_destination_encoding): new method
  Encoding::Converter#destination_encoding.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18625 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 14:28:10 +00:00
akr
87779b507e * transcode_data.h (rb_econv_result_t): change enumeration
constant's prefix: transcode_ to econv_.

* transcode.c: follow the constant prefix change.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 13:45:54 +00:00
akr
a6daaf16b0 * transcode.c (econv_init): accept Encoding object as source_encoding
and destination_encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18623 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 12:35:19 +00:00
akr
d46728ec0e * encoding.c (rb_filesystem_encoding): use default external encoding
for Unix.

* dir.c (dir_initialize): don't cache fs_encoding.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18622 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 12:11:14 +00:00
akr
db30661aa9 * transcode.c (rb_econv_open_by_transcoder_entries): renamed from
rb_trans_open_by_transcoder_entries.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18621 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 11:59:47 +00:00
akr
5a4844ed6c * transcode.c (econv_max_output): removed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 11:56:55 +00:00
akr
be51387d55 * tool/transcode-tblgen.rb: check unexpected actions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18619 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 11:53:47 +00:00
akr
bd9bd52954 * transcode_data.h (transcode_invalid_byte_sequence): renamed from
transcode_invalid_input.
  (transcode_destination_buffer_full): renamed from transcode_obuf_full.
  (transcode_source_buffer_empty): renamed from transcode_ibuf_empty.
  (rb_econv_result_t): renamed from rb_trans_result_t.
  (rb_econv_elem_t): renamed from rb_trans_elem_t.
  (rb_econv_t): renamed from rb_trans_t.

* transcode.c (UNIVERSAL_NEWLINE_DECODER): renamed from
  UNIVERSAL_NEWLINE.
  (CRLF_NEWLINE_ENCODER): renamed from CRLF_NEWLINE.
  (CR_NEWLINE_ENCODER): renamed from CR_NEWLINE.
  (rb_econv_open): renamed from rb_trans_open.
  (rb_econv_close): renamed from rb_trans_close.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18618 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 11:31:27 +00:00
akr
dd92751104 * transcode.c (load_transcoder): unused function removed.
(rb_transcoding_open): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18617 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 10:42:28 +00:00
akr
7db5e184c2 add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18616 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 10:38:41 +00:00