[#46049] [ruby-trunk - Feature #6590] Dealing with bigdecimal, etc gems in JRuby — "mrkn (Kenta Murata)" <muraken@...>
4 messages
2012/07/01
[#46078] [ruby-trunk - Feature #2565] adding hooks for better tracing — "mame (Yusuke Endoh)" <mame@...>
4 messages
2012/07/01
[#46108] Re: [ruby-trunk - Feature #2565] adding hooks for better tracing
— Aaron Patterson <tenderlove@...>
2012/07/02
On Mon, Jul 02, 2012 at 03:06:59AM +0900, mame (Yusuke Endoh) wrote:
[#46127] [ruby-trunk - Feature #2565] adding hooks for better tracing — "vo.x (Vit Ondruch)" <v.ondruch@...>
4 messages
2012/07/03
[#46160] [ruby-trunk - Feature #6693][Open] Don't warn for unused variables starting with _ — "marcandre (Marc-Andre Lafortune)" <ruby-core@...>
15 messages
2012/07/04
[#46163] [ruby-trunk - Feature #6695][Open] Configuration for Thread/Fiber creation — "ko1 (Koichi Sasada)" <redmine@...>
7 messages
2012/07/04
[#46172] [ruby-trunk - Feature #6697][Open] [PATCH] Add Kernel#Symbol conversion method like String(), Array() etc. — "madeofcode (Mark Dodwell)" <mark@...>
7 messages
2012/07/04
[#46236] [ruby-trunk - Bug #6704][Open] Random core dump — "trans (Thomas Sawyer)" <transfire@...>
7 messages
2012/07/07
[#46248] building ruby-1.9.3-p194 on AIX 6.1 TL05 SP06 — Perry Smith <pedzsan@...>
I am just now starting to debug this but hoped someone has already =
10 messages
2012/07/08
[#46251] Re: building ruby-1.9.3-p194 on AIX 6.1 TL05 SP06
— Yutaka Kanemoto <kinpoco@...>
2012/07/08
Hi Perry
[#46252] Re: building ruby-1.9.3-p194 on AIX 6.1 TL05 SP06
— Perry Smith <pedzsan@...>
2012/07/08
[#46254] Re: building ruby-1.9.3-p194 on AIX 6.1 TL05 SP06
— Perry Smith <pedzsan@...>
2012/07/08
[#46259] Re: building ruby-1.9.3-p194 on AIX 6.1 TL05 SP06
— Perry Smith <pedzsan@...>
2012/07/09
[#46266] Re: building ruby-1.9.3-p194 on AIX 6.1 TL05 SP06
— Yutaka Kanemoto <kinpoco@...>
2012/07/09
Hi Perry,
[#46262] [ruby-trunk - Feature #6710][Open] new special binding specifier :isolated — "ko1 (Koichi Sasada)" <redmine@...>
8 messages
2012/07/09
[#46276] Lambdaification of Method Calls — Robert Klemme <shortcutter@...>
Hi,
3 messages
2012/07/09
[#46320] [ruby-trunk - Feature #6721][Open] Object#yield_self — "alexeymuranov (Alexey Muranov)" <redmine@...>
25 messages
2012/07/11
[#46339] [ruby-trunk - Bug #6724][Open] waaaaaaant! ( — "zenspider (Ryan Davis)" <redmine@...>
11 messages
2012/07/11
[#46537] Re: [ruby-trunk - Bug #6724][Open] waaaaaaant! (
— Aaron Patterson <tenderlove@...>
2012/07/17
On Thu, Jul 12, 2012 at 08:58:36AM +0900, zenspider (Ryan Davis) wrote:
[#46540] Re: [ruby-trunk - Bug #6724][Open] waaaaaaant! (
— Luis Lavena <luislavena@...>
2012/07/18
On Tue, Jul 17, 2012 at 6:27 PM, Aaron Patterson
[#46377] [ruby-trunk - Feature #6727][Open] Add Array#rest (with implementation) — "duckinator (Nick Markwell)" <nick@...>
25 messages
2012/07/13
[#46420] [ruby-trunk - Feature #6731][Open] add new method "Object.present?" as a counter to #empty? — "rogerdpack (Roger Pack)" <rogerpack2005@...>
4 messages
2012/07/14
[#46500] [ruby-trunk - Feature #6739][Open] One-line rescue statement should support specifying an exception class — Quintus (Marvin Gülker) <sutniuq@...>
22 messages
2012/07/15
[#46535] [ruby-trunk - Bug #6749][Open] rdoc of Time class (incorrect explanation of leap seconds) — "stomar (Marcus Stollsteimer)" <redmine@...>
7 messages
2012/07/17
[#46687] [ruby-trunk - Bug #6749] rdoc of Time class (incorrect explanation of leap seconds)
— "drbrain (Eric Hodel)" <[email protected]>
2012/07/23
[#46713] Re: [ruby-trunk - Bug #6749] rdoc of Time class (incorrect explanation of leap seconds)
— sto.mar@...
2012/07/24
Hi Eric,
[#46734] Re: [ruby-trunk - Bug #6749] rdoc of Time class (incorrect explanation of leap seconds)
— Eric Hodel <[email protected]>
2012/07/24
On Jul 23, 2012, at 11:52 PM, [email protected] wrote:
[#46763] Re: [ruby-trunk - Bug #6749] rdoc of Time class (incorrect explanation of leap seconds)
— sto.mar@...
2012/07/25
Am 24.07.2012 19:44, schrieb Eric Hodel:
[#46546] Fwd: [ruby-cvs:43609] ko1:r36433 (trunk): * thread.c (rb_thread_call_without_gvl2): added. — SASADA Koichi <ko1@...>
Hi,
4 messages
2012/07/18
[#46547] Re: Fwd: [ruby-cvs:43609] ko1:r36433 (trunk): * thread.c (rb_thread_call_without_gvl2): added.
— Eric Wong <normalperson@...>
2012/07/18
SASADA Koichi <[email protected]> wrote:
[#46553] [ruby-trunk - Feature #2565] adding hooks for better tracing — "tenderlovemaking (Aaron Patterson)" <aaron@...>
4 messages
2012/07/18
[#46564] Ruby under CI - Windows — Luis Lavena <luislavena@...>
Hello,
6 messages
2012/07/20
[#46574] [ruby-trunk - Feature #6762][Open] Control interrupt timing — "ko1 (Koichi Sasada)" <redmine@...>
39 messages
2012/07/20
[#46575] [ruby-trunk - Feature #6762] Control interrupt timing
— "ko1 (Koichi Sasada)" <redmine@...>
2012/07/20
[#50542] Re: [ruby-trunk - Feature #6762][Open] Control interrupt timing
— Brent Roman <brent@...>
2012/12/03
I was suggesting "interruptible" as a better alternative for
[#46577] [ruby-trunk - Feature #6763][Open] Introduce Flonum technique to speedup floating computation on th 64bit environment — "ko1 (Koichi Sasada)" <redmine@...>
7 messages
2012/07/20
[#46586] [ruby-trunk - Bug #6764][Open] IO#read(size, buf) causes can't set length of shared string in trunk (2.0.0dev) — "nahi (Hiroshi Nakamura)" <nakahiro@...>
4 messages
2012/07/21
[#46641] [ruby-trunk - Bug #6780][Open] cannot compile zlib module, when cross-compiling. — "jinleileiking (lei king)" <jinleileiking@...>
14 messages
2012/07/23
[#46686] [ruby-trunk - Bug #6784][Open] Test failures related to numeric with x64 mingw — "h.shirosaki (Hiroshi Shirosaki)" <h.shirosaki@...>
5 messages
2012/07/23
[#46741] [ruby-trunk - Bug #6789][Open] parse.y compilation error due not updated id.h — "luislavena (Luis Lavena)" <luislavena@...>
8 messages
2012/07/24
[#46742] [ruby-trunk - Bug #6789][Assigned] parse.y compilation error due not updated id.h
— "luislavena (Luis Lavena)" <luislavena@...>
2012/07/24
[#46744] [ruby-trunk - Bug #6791][Open] ext/js on/generator/generator.c fails to compile on nightly build (AIX 6.1) — "pedz (Perry Smith)" <pedz@...>
4 messages
2012/07/25
[#46752] Re: [ruby-trunk - Bug #6791][Open] ext/js on/generator/generator.c fails to compile on nightly build (AIX 6.1)
— Yutaka Kanemoto <kinpoco@...>
2012/07/25
Hi Perry,
[#46772] Ruby 1.9.3 release? — Charles Oliver Nutter <headius@...>
JRuby will soon release 1.7.0pre2, the second preview of 1.7. Perhaps
6 messages
2012/07/25
[#46847] Re: Ruby 1.9.3 release?
— "NARUSE, Yui" <naruse@...>
2012/07/29
(2012/07/26 7:07), Charles Oliver Nutter wrote:
[#46864] Re: Ruby 1.9.3 release?
— Luis Lavena <luislavena@...>
2012/07/29
On Sat, Jul 28, 2012 at 10:59 PM, NARUSE, Yui <[email protected]> wrote:
[#46792] [ruby-trunk - Bug #6799][Open] Digest::*.hexdigest returns an ASCII-8BIT String — "Eregon (Benoit Daloze)" <redmine@...>
11 messages
2012/07/26
[#46832] [ruby-trunk - Bug #6807][Open] Can't compile ruby without ruby — "devcurmudgeon (Paul Sherwood)" <storitel@...>
13 messages
2012/07/28
[#46834] [ruby-trunk - Feature #6808][Open] Implicit index for enumerations — "trans (Thomas Sawyer)" <transfire@...>
15 messages
2012/07/28
[#46838] [ruby-trunk - Bug #6810][Open] `module A::B; end` is not equivalent to `module A; module B; end; end` with respect to constant lookup (scope) — "alexeymuranov (Alexey Muranov)" <redmine@...>
17 messages
2012/07/28
[#46854] [ruby-trunk - Feature #6811][Open] File, Dir and FileUtils should have bang-versions of singleton methods that fails silently — "prijutme4ty (Ilya Vorontsov)" <prijutme4ty@...>
7 messages
2012/07/29
[#46896] (Half-baked DRAFT) new `require' framework — SASADA Koichi <ko1@...>
Hi,
22 messages
2012/07/31
[#46897] Re: (Half-baked DRAFT) new `require' framework
— =?KOI8-R?B?4NLJyiDzz8vPzM/X?= <funny.falcon@...>
2012/07/31
2012/7/31 SASADA Koichi <[email protected]>
[#46899] Re: (Half-baked DRAFT) new `require' framework
— Alex Young <alex@...>
2012/07/31
On 31/07/12 13:29, SASADA Koichi wrote:
[#46902] Re: (Half-baked DRAFT) new `require' framework
— Trans <transfire@...>
2012/07/31
On Tue, Jul 31, 2012 at 12:07 PM, Alex Young <[email protected]> wrote:
[#46903] Re: (Half-baked DRAFT) new `require' framework
— Clifford Heath <clifford.heath@...>
2012/07/31
On 01/08/2012, at 5:59 AM, Trans wrote:
[#47040] Re: (Half-baked DRAFT) new `require' framework
— SASADA Koichi <ko1@...>
2012/08/07
(2012/07/31 21:29), SASADA Koichi wrote:
[#47054] Re: (Half-baked DRAFT) new `require' framework
— Rocky Bernstein <rockyb@...>
2012/08/07
If one is considering importing archive files like zip, tar, jar, or gem, I
[#47056] Re: (Half-baked DRAFT) new `require' framework
— Charles Oliver Nutter <headius@...>
2012/08/07
On Tue, Aug 7, 2012 at 8:48 AM, Rocky Bernstein <[email protected]> wrote:
[ruby-core:46158] [ruby-trunk - Feature #6612] Add streaming inflate and deflate to Zlib
From:
"nobu (Nobuyoshi Nakada)" <nobu@...>
Date:
2012-07-04 02:35:38 UTC
List:
ruby-core #46158
Issue #6612 has been updated by nobu (Nobuyoshi Nakada).
Don't modify a String object with realloc().
It causes a crash when CALC_EXACT_MALLOC_SIZE is set.
----------------------------------------
Feature #6612: Add streaming inflate and deflate to Zlib
https://bugs.ruby-lang.org/issues/6612#change-27778
Author: drbrain (Eric Hodel)
Status: Open
Priority: Normal
Assignee:
Category: ext
Target version: 2.0.0
=begin
Currently there is no way to control the inflate output size of a deflate stream from ruby. For example, 50MB of "0"s compress to just under 50KB:
ruby -rzlib -e 'p Zlib.deflate("0" * 50_000_000, Zlib::BEST_COMPRESSION).length' #=> 48611
When inflating this, 50MB are allocated which is undesirable.
The attached patch allows Zlib::Inflate#inflate, Zlib::Deflate#deflate, Zlib::ZStream#finish and other methods that end up calling zstream_expand_buffer or zstream_detach_buffer to be called with a block which gives the user more control over the amount of memory allocated in their process. (A fixed maximum chunk size of 16384 bytes is used in this patch.)
The new API looks like:
z = Zlib::Inflate.new
z.inflate deflate_string do |chunk|
# write chunk to output stream
# nil is returned from inflate
end
footer = z.finish
# flush buffer to output stream
Here's a comparison of resource usage:
$ dd if=/dev/zero of=/dev/stdout bs=1m count=1024 | gzip -c > 1G.gz
$ cat test.rb
require 'zlib'
gzipped = File.read '1G.gz'
z = Zlib::Inflate.new Zlib::MAX_WBITS + 32
z.inflate gzipped do |chunk|
# do nothing with chunk
# current ruby will ignore this block
end
z.finish
With existing ruby (trunk 35758) 1GB of memory is allocated:
$ /usr/bin/time -l ruby20 test.rb
3.86 real 3.24 user 0.61 sys
1080475648 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
263860 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
1 voluntary context switches
34 involuntary context switches
With the patch only 37MB are allocated:
$ /usr/bin/time -l ./ruby20 -I .ext/x86_64-darwin11.4.0 test.rb
3.47 real 3.43 user 0.03 sys
36724736 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
8981 page reclaims
0 page faults
0 swaps
0 block input operations
9 block output operations
0 messages sent
0 messages received
0 signals received
1 voluntary context switches
90 involuntary context switches
Some notes about this patch:
zstream_expand_buffer() yields each block of inflate (or deflate) output instead of expanding the buffer (as in non-block output). zstream_expand_buffer_into() is used to reduce duplication as well.
zstream_detach_buffer() yields the buffer and returns nil if a block was given. OBJ_INFECT was moved into zstream_detach_buffer() as well.
A new stream flag ZSTREAM_FLAG_GZFILE is added to prevent a block given to a gzip method that invokes zstream_expand_buffer() or zstream_detach_buffer() from yielding.
To ensure proper handling of ((|z->buf|)) such as resetting the buffer after yielding a chunk, rb_protect is used in zstream_expand_buffer and zstream_run. In zstream_run, the for loop has been extracted to zstream_run_loop.
In zstream_run_loop, Zlib::BufError is no longer raised when flushing (finishing) the stream. This allows the stream to be flushed into one string when the inflate block is interrupted.
=end
--
http://bugs.ruby-lang.org/