From: Eric Wong <normalperson@...>
Date: 2012-11-21T12:43:16+09:00
Subject: [ruby-core:49794] Re: [ruby-trunk - Feature #6047][Assigned] read_all: Grow buffer exponentially in generic case

Martin Bosslet <Martin.Bosslet@googlemail.com> wrote:
> In the general case, read_all grows its buffer linearly by just the
> amount that is currently read from the underlying source. This results
> in a linear number of reallocs, It might turn out beneficial if the
> buffer were grown exponentially by multiplying with a constant factor
> (e.g. 1.5 or 2), thus resulting in only a logarithmic numver of
> reallocs.

I think growing the buffer exponentially makes sense.

I would enforce a hard limit (probably <= 8 MB) for each growth,
to:

1) discourage read_all() for large files, it's very wasteful and
   usually hurts performance

2) prevent memory exhaustion for edge cases (especially on 32-bit)