summaryrefslogtreecommitdiff
path: root/include/ruby/encoding.h
AgeCommit message (Collapse)Author
2014-06-02encoding.c: move ruby_encoding_index stuffnobu
* encoding.c: move `ruby_encoding_index` stuff from include/ruby/encoding.h to hide the extra field. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-02encoding.h: constify rb_encodingnobu
* include/ruby/encoding.h: constify `rb_encoding` itself, not only arguments. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46322 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-01constify rb_encoding and OnigEncodingnobu
* include/ruby/encoding.h: constify `rb_encoding` arguments. * include/ruby/oniguruma.h: constify `OnigEncoding` arguments. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-06transcode.c: add rb_econv_appendnobu
* transcode.c (rb_econv_append): new function to append a string data with converting its encoding. split from rb_econv_substr_append. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42853 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-03string.c: rb_enc_str_new_cstrnobu
* string.c (rb_enc_str_new_cstr): new function to create a string from the C-string pointer with the specified encoding. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42811 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-08-30encoding.h: fix conflictsnobu
* include/ruby/encoding.h (rb_{ascii8bit,utf8,usascii}_encindex): get rid of conflict with macros defined in internal.h. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42734 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-08-11* include/ruby/encoding.h: Reduce ENCODING_INLINE_MAX to 127 as thischarliesome
should be sufficient to represent all the encodings Ruby supports. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42508 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-08-05* include/ruby/encoding.h: document which user flags are used bycharliesome
ENCODING_MASK for better greppability git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-07-20encoding.c: add rb_enc_code_to_mbclennobu
* encoding.c (rb_enc_code_to_mbclen): add new function which returns mbclen from codepoint like as rb_enc_codelen() but 0 for invalid char. * include/ruby/encoding.h (rb_enc_code_to_mbclen): declaration and shortcut macro. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42077 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-07-02encoding.c: validate indexnobu
* encoding.c (rb_enc_set_index, rb_enc_associate_index): validate argument encoding index. * include/ruby/encoding.h (ENCODING_SET): use rb_enc_set_index() instead of setting inlined bits directly. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41739 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-05defines.h: RUBY_SYMBOL_EXPORT_{BEGIN,END}nobu
* include/ruby/defines.h (RUBY_SYMBOL_EXPORT_{BEGIN,END}): visibility control macros. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40122 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-11* parse.y (rb_check_id_cstr): new function to check if ID isnobu
registered with NUL-terminated C string. * sprintf.c (rb_str_format): avoid inadvertent symbol creation. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35300 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-10* error.c (rb_enc_raise): new function to raise an exception withnobu
the message in the given encoding. patched by now (Nikolai Weibull) at [ruby-core:41160]. [Feature #5650] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35283 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-03-12* encoding.c (rb_find_encoding): new function find encoding fromnobu
arbitrary object as a pointer to rb_encoding, and return NULL if not found. * io.c (io_encoding_set): just warn unsupported encodings, but not exception. [ruby-core:40726] [Bug #5567] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-01-25* file.c (rb_enc_path_next, rb_enc_path_skip_prefix)nobu
(rb_enc_path_last_separator, rb_enc_path_end) (ruby_enc_find_basename, ruby_enc_find_extname): encoding-aware path handling functions. * file.c (rb_home_dir, file_expand_path, rb_realpath_internal) (rb_file_s_basename, rb_file_dirname, rb_file_s_extname) (rb_file_join): should respect the encodings of arguments than file system encoding. [ruby-dev:45145] [Bug #5919] * dir.c (check_dirname, ruby_glob0): ditto. * ext/pathname/pathname.c (path_sub_ext): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34372 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-12-04Introduce NEED_READCONV and NEED_WRITECONV to replace universal newline ↵luislavena
decorator Use CRLF only when required to improve file reading and writing under Windows. Patch by Hiroshi Shirosaki. [ruby-core:40706] [Feature #5562] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33937 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-11-07* include/ruby/encoding.h (ECONV_NEWLINE_DECORATOR_READ_MASK,usa
ECONV_NEWLINE_DECORATOR_WRITE_MASK): new macro. * io.c (rb_io_extract_modeenc, pipe_open, prep_stdio, argf_next_argv): set TEXTMODE_NEWLINE_DECORATOR_ON_WRITE for textmode on creating IO if the flag is available. * io.c (make_writeconv): drop decorators for reading. * io.c (make_readconv): drop decorators for writing. * io.c (do_writeconv): existing writeconv is not the condition to raise ArgumentError. should check textmode or not. * test/ruby/test_io_m17n.rb (TestIO_M17N#test_{cr,lf,crlf}_decorator_on_stdout): test above changes. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-04-26* io.c (validate_enc_binmode, rb_io_extract_modeenc): set newlinenobu
decorator according to open mode. * transcode.c (rb_econv_prepare_options): new function, to prepare econv options with newline flags. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31355 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-04-26* include/ruby/encoding.h (ECONV_NEWLINE_DECORATOR_MASK): add.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-04-05* include/ruby/encoding.h: parenthesize macro arguments.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31241 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-02-03* include/ruby/encoding.h (rb_enc_step_back): cast 4th argument 'n'naruse
as int because Ruby usually treats length value as long but onigenc_step_back's 4th argument is int. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@30768 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-01-30* include/ruby/encoding.h (rb_enc_step_back): new function to stepnobu
back n characters. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@30728 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-10-12* numeric.c (rb_enc_uint_chr): split from int_chr.naruse
* numeric.c (int_chr): use rb_enc_uint_chr. * include/ruby/encoding.h (rb_enc_uint_chr): added. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29446 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-08-11* include/ruby/encoding.h (rb_char_to_option_kcode): used innobu
ripper. * node.h (rb_reserved_word): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28970 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-07-21* configure.in (XCFLAGS): use -fvisibility=hidden if possible.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-05-27* include/ruby/encoding.h (rb_filesystem_encindex): defined.naruse
* include/ruby/encoding.h (rb_locale_encindex): ditto. * encoding.c (rb_filesystem_encindex): remove static. * encoding.c (rb_locale_encindex): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-05-25* include/ruby/vm.h, include/ruby/encoding.h: add externalnobu
linkage. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-03-16* include/ruby/encoding.h (rb_str_conv_enc_opts): missing prototype.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26953 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-03-16* include/ruby/encoding.h (ENC_TO_ENCINDEX): suppress warnings.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26950 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-29* encoding.c (rb_enc_unicode_p): defined.naruse
Returns 1 when the encoding is Unicode series other than UTF-7 else 0. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-14* debug.c, parse.y: fixed types.nobu
* node.h (nd_line): limit to int. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24924 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-05-20* encoding.c (rb_enc_fast_mbclen): faster mbclen for strings knownmatz
to be valid. * string.c (enc_strlen): coderange specified version of rb_enc_strlen(). use rb_enc_fast_mbclen() if coderange is 7bit or valid. * string.c (str_gsub): use rb_enc_fast_mbclen(). * string.c (rb_str_reverse, rb_str_split_m, rb_str_each_char, scan_once): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23495 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-05-19* encoding.c (rb_enc_codepoint_len): combine rb_enc_codepoint()matz
and rb_enc_codelen() in one function to reduce calls. * encoding.c (rb_enc_codepoint): compatibility function. * sprintf.c (rb_str_format): use rb_enc_codepoint_len(). * string.c (rb_str_inspect, rb_str_upcase_bang, rb_str_downcase_bang, rb_str_capitalize_bang, rb_str_swapcase_bang, trnext, tr_trans, rb_str_delete_bang, rb_str_squeeze_bang, rb_str_count, rb_str_split_m, rb_str_each_line, rb_str_each_codepoint, rb_str_lstrip_bang, sym_printable): ditto. * transcode.c (make_econv_exception): use rb_enc_mbc_to_codepoint() git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-05-07* string.c, include/ruby/encoding.h: fixed types.nobu
* include/ruby/encoding.h (rb_enc_nth): long is used for index. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23358 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-03-15* include/ruby/ruby.h ({RSTRING,RBIGNUM}_EMBED_LEN_MAX): made int.nobu
* include/ruby/ruby.h (OBJ_{TAINTED,UNTRUSTED,FROZEN}): return int. * include/ruby/encoding.h (ENC_CODERANGE): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-03-12* array.c, bignum.c, dln.c, error.c, gc.c, io.c, marshal.c,nobu
numeric.c, pack.c, strftime.c, string.c, thread.c, transcode.c, transcode_data.h, util.c, variable.c, vm_dump.c, include/ruby/encoding.h, missing/crypt.c, missing/vsnprintf.c: suppress VC type warnings. [ruby-core:22726] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-03-12* include/ruby/encoding.h: suppress warnings of overflow.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22905 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-03-05* include/ruby/encoding.h (enc_initialized_p): ENC_UNINITIALIZEDnobu
is not defined already. * include/ruby/encoding.h (ENC_FROM_ENCINDEX): removed because depending on a static variable. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-26* io.c (fptr_finalize): close the IO object even if finish_writeconv orakr
flush is failed. (finish_writeconv): don't raise. return errno or exception. (finish_writeconv_arg): removed. (finish_writeconv_sync): follow finish_writeconv change. * transcode.c (rb_econv_make_exception): new function. * include/ruby/encoding.h (rb_econv_make_exception): declared. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21059 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-21* ext/zlib/zlib.c: remove obsolete prototype macros.matz
* ext/zlib/zlib.c (struct gzfile): add encoding field to gzfile structure. * ext/zlib/zlib.c (rb_gzreader_getc): now works on characters. * ext/zlib/zlib.c (rb_gzreader_getbyte): new method to retrieve single byte. * ext/zlib/zlib.c (rb_gzreader_readbyte): ditto. * ext/zlib/zlib.c (rb_gzreader_each_byte): renamed from each_char * ext/zlib/zlib.c (rb_gzreader_ungetc): allow unget strings. * ext/zlib/zlib.c (rb_gzreader_ungetbyte): renamed from ungetc. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-20* include/ruby/ruby.h (ExportStringValue): new macro to convertmatz
string in internal encoding to external to export. * string.c (rb_str_export): new function to do conversion to external encoding. * ext/sdbm/init.c: encoding conversion support. * ext/dbm/dbm.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-18* string.c (rb_external_str_new): a new function to convert frommatz
external encoding to internal encoding. if something went wrong, it returns a string with the external encoding. * string.c (rb_external_str_new_with_enc): same as above besides you can specify the source encoding. * ruby.c (ruby_set_argv): use rb_external_str_new() * ruby.c (set_arg0, ruby_script): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19834 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-07* encoding.c (rb_default_internal_encoding): merged a patch frommatz
Michael Selig <michael.selig at fs.com.au> in [ruby-core:18985]. * io.c (rb_io_ext_int_to_encs): ditto. * ruby.c (proc_options): support default internal encoding in -E option. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-26* include/ruby/encoding.h (rb_str_encode): renamed fromakr
rb_str_transcode. [ruby-dev:36593] (rb_econv_has_convpath_p): renamed from rb_transcode_convertible. * transcode.c: follow the renaming. * io.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19589 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-26 * transcode.c, include/ruby/encoding.c (rb_transcode_convertible):usa
new function. checking the existance of converter. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-22 * include/ruby/encoding.h (enum): ISO C forbids comma at end ofshyouhei
enumerator list git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19453 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-18* grapheme cluster implementation reverted. [ruby-dev:36375]akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-17* string.c (rb_str_casecmp): don't use rb_enc_codepoint.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19398 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-16* include/ruby/oniguruma.h (OnigEncodingTypeST): add precise_retakr
argument for mbc_to_code. (ONIGENC_MBC_TO_CODE): provide NULL for precise_ret. (ONIGENC_MBC_PRECISE_CODEPOINT): defined. * include/ruby/encoding.h (rb_enc_mbc_precise_codepoint): defined. * regenc.h (onigenc_single_byte_mbc_to_code): precise_ret argument added. (onigenc_mbn_mbc_to_code): ditto. * regenc.c (onigenc_single_byte_mbc_to_code): precise_ret argument added. (onigenc_mbn_mbc_to_code): ditto. * string.c (count_utf8_lead_bytes_with_word): removed. (str_utf8_nth): removed. (str_utf8_offset): removed. (str_strlen): UTF-8 codepoint oriented optimization removed. (rb_str_substr): ditto. (enc_succ_char): use rb_enc_mbc_precise_codepoint. (enc_pred_char): ditto. (rb_str_succ): ditto. * encoding.c (rb_enc_ascget): check length with rb_enc_mbc_precise_codepoint. (rb_enc_codepoint): use rb_enc_mbc_precise_codepoint. * regexec.c (string_cmp_ic): add text_end argument. (match_at): check end of character after exact string matches. * enc/utf_8.c (graphme_table): defined for extended graphme cluster boundary. (grapheme_cmp): defined. (get_grapheme_properties): defined. (grapheme_boundary_p): defined. (MAX_BYTES_LENGTH): defined. (comb_char_enc_len): defined. (mbc_to_code0): extracted from mbc_to_code. (mbc_to_code): use mbc_to_code0. (left_adjust_combchar_head): defined. (utf_8): use a extended graphme cluster as a unit. * enc/unicode.c (onigenc_unicode_mbc_case_fold): use ONIGENC_MBC_PRECISE_CODEPOINT to extract codepoints. (onigenc_unicode_get_case_fold_codes_by_str): ditto. * enc/euc_jp.c (mbc_to_code): follow mbc_to_code field change. use onigenc_mbn_mbc_to_code. * enc/shift_jis.c (mbc_to_code): ditto. * enc/emacs_mule.c (mbc_to_code): ditto. * enc/gbk.c (gbk_mbc_to_code): follow mbc_to_code field and onigenc_mbn_mbc_to_code change. * enc/cp949.c (cp949_mbc_to_code): ditto. * enc/big5.c (big5_mbc_to_code): ditto. * enc/euc_tw.c (euctw_mbc_to_code): ditto. * enc/euc_kr.c (euckr_mbc_to_code): ditto. * enc/gb18030.c (gb18030_mbc_to_code): ditto. * enc/utf_32be.c (utf32be_mbc_to_code): follow mbc_to_code field change. * enc/utf_16be.c (utf16be_mbc_to_code): ditto. * enc/utf_32le.c (utf32le_mbc_to_code): ditto. * enc/utf_16le.c (utf16le_mbc_to_code): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-13* include/ruby/oniguruma.h (OnigEncodingTypeST): add end argument forakr
left_adjust_char_head. (ONIGENC_LEFT_ADJUST_CHAR_HEAD): add end argument. (onigenc_get_left_adjust_char_head): ditto. * include/ruby/encoding.h (rb_enc_left_char_head): add end argument. * regenc.h (onigenc_single_byte_left_adjust_char_head): ditto. * regenc.c (onigenc_get_right_adjust_char_head): follow the interface change. (onigenc_get_right_adjust_char_head_with_prev): ditto. (onigenc_get_prev_char_head): ditto. (onigenc_step_back): ditto. (onigenc_get_left_adjust_char_head): ditto. (onigenc_single_byte_code_to_mbc): ditto. * re.c: ditto. * string.c: ditto. * io.c: ditto. * regexec.c: ditto. * enc/euc_jp.c: ditto. * enc/cp949.c: ditto. * enc/shift_jis.c: ditto. * enc/gbk.c: ditto. * enc/big5.c: ditto. * enc/euc_tw.c: ditto. * enc/euc_kr.c: ditto. * enc/emacs_mule.c: ditto. * enc/gb18030.c: ditto. * enc/utf_8.c: ditto. * enc/utf_16le.c: ditto. * enc/utf_16be.c: ditto. * enc/utf_32le.c: ditto. * enc/utf_32be.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19334 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2025-05-18 05:40:00 +0000