Commit graph

279 commits

Author SHA1 Message Date
yugui
a50f85f6cb merges r23773 from trunk into ruby_1_9_1.
--
* transcode.c (enc_arg): default interanl encoding may not be set.
  [ruby-core:23932]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@23807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-06-21 10:36:42 +00:00
yugui
e4d2470612 merges r23682 from trunk into ruby_1_9_1.
--
Fix rdoc.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@23786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-06-21 09:15:16 +00:00
yugui
b371ce9d6a merges r21532 from trunk into ruby_1_9_1.
Add rdoc.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21568 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-01-15 15:40:21 +00:00
yugui
d68854c07f merges r21509 and r21512 from trunk into ruby_1_9_1.
* enc/trans/gb18030.trans, gb18030-tbl.rb:
  new Chinese GB18030 transcoding (from Yoshihiro Kambayashi)

* test/ruby/test_transcode.rb: added tests for the above
  (from Yoshihiro Kambayashi)

* transcode_data.h, transcode.c, tool/transcode_tblgen.rb:
  added support for GB18030-specific 4-byte sequences
  (with Yoshihiro Kambayashi)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21560 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-01-15 15:39:02 +00:00
yugui
d5251e8d25 merges r21488 from trunk into ruby_1_9_1.
Add dummy definition for fake rdoc.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21548 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-01-15 15:37:13 +00:00
yugui
3dbaf1ebf8 merges r21187 from trunk into ruby_1_9_1.
* transcode.c: Minor fixes and tweaks in documentation.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-30 11:05:23 +00:00
yugui
0c0804cf26 merges r21092 from trunk into ruby_1_9_1.
* transcode.c (make_econv_exception): show source and destination
  encoding.  [ruby-dev:37285]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21122 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-28 09:41:23 +00:00
yugui
d2165fe2bb merges r21059 from trunk into ruby_1_9_1.
* io.c (fptr_finalize): close the IO object even if finish_writeconv or
  flush is failed.
  (finish_writeconv): don't raise.  return errno or exception.
  (finish_writeconv_arg): removed.
  (finish_writeconv_sync): follow finish_writeconv change.

* transcode.c (rb_econv_make_exception): new function.

* include/ruby/encoding.h (rb_econv_make_exception): declared.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21069 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-26 11:16:07 +00:00
yugui
b7a1cb6251 merges r21047 from trunk into ruby_1_9_1.
* transcode.c (str_transcode0): set encoding when String#encode was
  given explicit but the same destination and source encoding.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21064 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-26 05:54:41 +00:00
yugui
1f161c1824 merges r21044 from trunk into ruby_1_9_1.
* transcode.c (sym_ignore): remove useless symbol.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-26 05:54:27 +00:00
yugui
32b66dcf90 merges r20991 from trunk into ruby_1_9_1.
* transcode.c (decorate_convpath): show type of escaping for
  xml_attr_quote or some conversions.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21042 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-25 09:58:27 +00:00
yugui
9c171e3d6b merges r20946 from trunk into ruby_1_9_1.
* io.c: rdoc for File::open and 1.9 feature in file modes.

* transcode.c: rdoc for String#encode

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_1_9_1@21017 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-25 09:54:17 +00:00
duerst
da1ffe490d * transcode.c: in transcode_search_path, elimintated a warning
on cygwin about pathlen potentially not being initialized


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19830 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-18 07:41:09 +00:00
matz
5ef37965fc * transcode.c (str_transcode0): String#encode without argument now
behave as if :undef => :replace, :invalid => :replace specified.

* transcode.c (rb_econv_prepare_opts): should preserve options in
  any case.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19818 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-17 09:13:04 +00:00
akr
c2b9d34f62 * transcode.c (transcode_restartable0): in_pos and out_pos never be
NULL, now.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-16 16:26:53 +00:00
matz
80cdf946f7 * transcode.c (str_transcode0): should not raise error when
default_internal is not set.  [ruby-core:19309]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-14 14:05:11 +00:00
nobu
a09a8ea86d * transcode.c (rb_econv_init_by_convpath): suppressed warnings.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19769 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-13 00:56:40 +00:00
matz
284f4c8431 * transcode.c (str_transcode0): String#encode() with no encoding
specified transcode the string into Encoding.default_internal.
  inspired by [ruby-core:19298].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19764 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-12 22:52:01 +00:00
mame
fc9c37dfc9 * transcode.c (rb_econv_add_coverter): remove unnecessary NULL check.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19733 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-09 15:12:20 +00:00
akr
a3bebd5564 * include/ruby/encoding.h (rb_str_encode): renamed from
rb_str_transcode.  [ruby-dev:36593]
  (rb_econv_has_convpath_p): renamed from rb_transcode_convertible.

* transcode.c: follow the renaming.

* io.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19589 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-26 10:35:00 +00:00
matz
3deb806a06 * error.c (Init_Exception): introduce EncodingError which is a
superclass for all encoding related exception classes,
  e.g. Encoding::CompatibilityError.  [ruby-dev:36371]

* transcode.c (Init_transcode): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19570 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-26 03:53:11 +00:00
usa
e76b1a11ca * transcode.c, include/ruby/encoding.c (rb_transcode_convertible):
new function. checking the existance of converter.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-26 02:07:19 +00:00
akr
2a608fed88 update rdoc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-25 17:42:52 +00:00
akr
d7dbdef4e5 * transcode.c (rb_eUndefinedConversionError): renamed from
rb_eConversionUndefinedError.
  (rb_eConverterNotFoundError): renamed from rb_eNoConverterError.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19554 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-25 12:47:30 +00:00
akr
fee3bb664a update rdoc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-25 11:42:49 +00:00
akr
a7dd79c1b4 update rdoc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-24 16:02:37 +00:00
akr
2cbc7b69b7 * transcode.c (enc_arg): must take pointer argument to avoid GC
problem.  StringValueCStr modifies the argument and it should be
  preserved while the string StringValueCStr returns is used.
  Since the string is used by caller, the modified argument should be
  hold by caller.  Actually
    GC.stress = true
    def (o=Object.new).to_str()
      "universal"+"_newline"
    end                         
    "\u3042".encode(o, "")' 
  causes curious warning:
    rb_define_const: invalid name `' for constant



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19408 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-18 09:29:01 +00:00
matz
d5ec36c486 * transcode.c: add "Error" suffix for Encoding exception classes.
a patch from Tadashi Saito <shiba at mail2.accsnet.ne.jp> in
  [ruby-dev:36346].

* encoding.c (Init_Encoding): rename EncodingCompatibilityError to
  Encoding::CompatibilityError.  [ruby-dev:36366]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-18 08:57:33 +00:00
matz
45ee7d2e78 * transcode.c (enc_arg): revert last change. too early exception
raising.

* transcode.c (enc_arg): need not to take pointer argument.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19406 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-18 08:27:17 +00:00
naruse
bad9a9ad0d * transcode.c (enc_arg): raise exception when unknown encoding is
given.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-16 16:18:32 +00:00
akr
6eb3843b17 * transcode_data.h (STR1_LENGTH): defined.
(makeSTR1LEN): defined.

* tool/transcode-tblgen.rb: use makeSTR1LEN.  generate STR1 for 4 to
  259 bytes.

* transcode.c (rb_transcoding): new field: output_index.
  (transcode_restartable0): use STR1_LENGTH.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19366 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-15 15:42:30 +00:00
akr
e0f064cf3c * transcode.c (econv_opts): don't use to_sym.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-15 14:38:12 +00:00
akr
c767be3039 * transcode_data.h: return output functions ssize_t.
* transcode.c (transcode_restartable0): don't need to cast the result
  of output functions.

* enc/trans/newline.trans: follow the type change.

* enc/trans/escape.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-14 18:35:17 +00:00
akr
a3c8c0adec * transcode_data.h: output function takes output buffer size.
* transcode.c: give output buffer size for output functions.

* enc/trans/newline.trans: follow the type change.

* enc/trans/escape.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-14 18:06:20 +00:00
nobu
2f67b43aec * transcode.c (str_encode): returns duplicated string if nothing
changed.  [ruby-core:18578]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19349 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-14 17:37:35 +00:00
akr
0a3fc5be05 * transcode.c (rb_transcoding): align state field.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19326 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-13 09:52:02 +00:00
akr
d4c7473841 * transcode.c (DECORATOR_P): renamed from SUPPLEMENTAL_CONVERSION.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19325 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-13 09:21:34 +00:00
akr
8d7e774f5f * io.c (make_writeconv): use rb_enc_name.
(make_readconv): ditto.

* transcode.c (rb_econv_open_opts): ditto.
  (econv_args): ditto.
  (decorate_convpath): ditto.
  (econv_set_replacement): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19324 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-13 09:06:51 +00:00
akr
2cc9b488a0 * transcode.c (rb_econv_prepare_opts): raise ArgumentError if
a broken string is specified as a replacement.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 18:45:44 +00:00
akr
7e58885f0a * transcode.c (rb_econv_prepare_opts): initialize *opt and return 0
if opthash is nil. 


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19312 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 17:53:26 +00:00
akr
41d3a01486 * enc/trans/escape.trans: transcoder name renamed to use underscore.
* transcode.c: follow the renaming.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19311 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 17:30:07 +00:00
akr
248acf0025 * transcode.c (rb_econv_init_by_convpath_i): new function.
(rb_econv_init_by_convpath): new function.
  (econv_init): use rb_econv_init_by_convpath.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 17:13:04 +00:00
akr
cf86176338 * transcode.c (econv_args): take snamev_p and dnamev_p to prevent
possible GC problem.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 16:13:02 +00:00
akr
aa423b9311 * transcode.c (enc_arg): take arg as VALUE * to prevent GC problem.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19307 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 15:39:46 +00:00
akr
c0bd38eb94 * transcode.c (rb_econv_add_transcoder_at): fix loop condition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19306 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 14:20:27 +00:00
akr
2d80b1695d * transcode.c (decorator_names): extracted from rb_econv_open.
(rb_econv_open): use decorator_names.
  (econv_args): extracted from econv_init.
  (econv_init): use econv_args.
  (decorate_convpath): new function.
  (search_convpath_i): new function.
  (econv_s_search_convpath): new method.
  (Init_transcode): new method defined.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 13:53:42 +00:00
akr
89a5c998c7 * transcode.c (rb_econv_t): new field: num_allocated.
(rb_econv_alloc): extracted from
  rb_econv_open_by_transcoder_entries.
  (rb_econv_add_transcoder_at): extracted from rb_econv_decorate_at
  and generalized 
  (rb_econv_open_by_transcoder_entries): use rb_econv_alloc and
  rb_econv_add_transcoder_at.
  (rb_econv_add_converter): extracted from rb_econv_decorate_at.
  (rb_econv_decorate_at): use rb_econv_add_converter.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 12:58:23 +00:00
akr
19438eee8b * transcode.c (rb_econv_t): new field: started.
(rb_econv_open_by_transcoder_entries): initialize started field.
  (rb_econv_convert): set started field.
  (rb_econv_insert_output): ditto.
  (rb_econv_decorate_at): check started field instead of num_finished.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19303 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-12 12:23:25 +00:00
akr
10ddce3833 * transcode.c (make_encoding): new function.
(make_encobj): new function.
  (econv_s_asciicompat_encoding): use make_encoding.
  (rb_econv_open_exc): use SUPPLEMENTAL_CONVERSION.
  (econv_convpath): use encoding object in the result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-10 17:26:49 +00:00
akr
4e93ace24e * transcode.c (econv_convpath): new method.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-10 17:15:08 +00:00