[#47787] Ruby Parallelism — Miguel Palhas <mpalhas@...>
Greetings
4 messages
2012/10/01
[#47790] [ruby-trunk - Bug #7097][Open] Thread locals don't work inside Enumerator — "tenderlovemaking (Aaron Patterson)" <aaron@...>
32 messages
2012/10/01
[#47791] [ruby-trunk - Bug #7097][Assigned] Thread locals don't work inside Enumerator
— "kosaki (Motohiro KOSAKI)" <kosaki.motohiro@...>
2012/10/01
[#47792] Re: [ruby-trunk - Bug #7097][Assigned] Thread locals don't work inside Enumerator
— Aaron Patterson <tenderlove@...>
2012/10/01
On Tue, Oct 02, 2012 at 03:05:17AM +0900, kosaki (Motohiro KOSAKI) wrote:
[#47798] Re: [ruby-trunk - Bug #7097][Assigned] Thread locals don't work inside Enumerator
— SASADA Koichi <ko1@...>
2012/10/01
(2012/10/02 3:12), Aaron Patterson wrote:
[#47800] Re: [ruby-trunk - Bug #7097][Assigned] Thread locals don't work inside Enumerator
— SASADA Koichi <ko1@...>
2012/10/01
(2012/10/02 8:22), SASADA Koichi wrote:
[#47802] Re: [ruby-trunk - Bug #7097][Assigned] Thread locals don't work inside Enumerator
— Aaron Patterson <tenderlove@...>
2012/10/02
On Tue, Oct 02, 2012 at 08:32:51AM +0900, SASADA Koichi wrote:
[#47807] Re: [ruby-trunk - Bug #7097][Assigned] Thread locals don't work inside Enumerator
— KOSAKI Motohiro <kosaki.motohiro@...>
2012/10/02
>> For example:
[#48291] [ruby-trunk - Bug #7097] Thread locals don't work inside Enumerator
— "tenderlovemaking (Aaron Patterson)" <aaron@...>
2012/10/25
[#48299] Re: [ruby-trunk - Bug #7097] Thread locals don't work inside Enumerator
— SASADA Koichi <ko1@...>
2012/10/26
I don't have any objection.
[#48300] Re: [ruby-trunk - Bug #7097] Thread locals don't work inside Enumerator
— Aaron Patterson <tenderlove@...>
2012/10/26
On Fri, Oct 26, 2012 at 02:40:53PM +0900, SASADA Koichi wrote:
[#47832] [ruby-trunk - Feature #7106][Open] FileUtils.touch should allow touching the symlink itself rather than the file the link points to — "cirrusthinking (Alessandro Diaferia)" <alessandro@...>
18 messages
2012/10/04
[#47841] [ruby-trunk - Bug #7109][Open] File.utime doesn't set nanoseconds — "bkabrda (Bohuslav Kabrda)" <bkabrda@...>
6 messages
2012/10/05
[#47842] Re: [ruby-trunk - Bug #7109][Open] File.utime doesn't set nanoseconds
— Tanaka Akira <akr@...>
2012/10/05
2012/10/5 bkabrda (Bohuslav Kabrda) <[email protected]>:
[#47847] [ruby-trunk - Bug #7110][Open] CGI: Add support for HTML5 <header> tag — "stomar (Marcus Stollsteimer)" <redmine@...>
16 messages
2012/10/05
[#47880] [ruby-trunk - Bug #7134][Open] Signal handling bug in Mac OS X — "auastro (Andy Kitchen)" <kitchen.andy+rubybug@...>
17 messages
2012/10/10
[#48918] [ruby-trunk - Bug #7134][Assigned] Signal handling bug in Mac OS X
— "mame (Yusuke Endoh)" <mame@...>
2012/11/05
[#47881] [ruby-trunk - Bug #7135][Open] GC bug in Ruby 1.9.3-p194? — "alexdowad (Alex Dowad)" <alexinbeijing@...>
21 messages
2012/10/10
[#47887] [ruby-trunk - Bug #7137][Open] Date.parse overly lenient when attempting to parse Monday? — "garysweaver (Gary Weaver)" <garysweaver@...>
12 messages
2012/10/10
[#47927] new ruby 1.9.3 maintainer — "U.Nakamura" <usa@...>
Hello everyone of the Ruby development community
6 messages
2012/10/12
[#47930] [ruby-trunk - Feature #7148][Open] Improved Tempfile w/o DelegateClass — "Glass_saga (Masaki Matsushita)" <glass.saga@...>
14 messages
2012/10/12
[#47963] [ruby-trunk - Bug #7154][Open] For whatever reason \s doesn't match \u00a0. — "t0d0r (Todor Dragnev)" <todor.dragnev@...>
5 messages
2012/10/13
[#47970] [ruby-trunk - Bug #7158][Open] require is slow in its bookkeeping; can make Rails startup 2.2x faster — "gregprice (Greg Price)" <price@...>
30 messages
2012/10/14
[#48504] [ruby-trunk - Bug #7158] require is slow in its bookkeeping; can make Rails startup 2.2x faster
— "h.shirosaki (Hiroshi Shirosaki)" <h.shirosaki@...>
2012/10/27
[#48505] Re: [ruby-trunk - Bug #7158] require is slow in its bookkeeping; can make Rails startup 2.2x faster
— SASADA Koichi <ko1@...>
2012/10/27
(2012/10/28 7:10), h.shirosaki (Hiroshi Shirosaki) wrote:
[#48728] [ruby-trunk - Bug #7158] require is slow in its bookkeeping; can make Rails startup 2.2x faster
— "tenderlovemaking (Aaron Patterson)" <aaron@...>
2012/11/02
[#48732] Re: [ruby-trunk - Bug #7158] require is slow in its bookkeeping; can make Rails startup 2.2x faster
— Masaya TARUI <tarui@...>
2012/11/02
Thank you for the careful work.
[#48032] [Backport93 - Backport #7174][Open] Advocating for backporting 36811 — "jonforums (Jon Forums)" <redmine@...>
8 messages
2012/10/16
[#48086] [Backport93 - Backport #7174][Assigned] Advocating for backporting 36811
— "usa (Usaku NAKAMURA)" <usa@...>
2012/10/19
[#48040] Should Hash#dup automatically rehash — Aaron Patterson <tenderlove@...>
Hi,
5 messages
2012/10/17
[#48072] [ruby-trunk - Bug #7184][Open] --disable-gems commandline parameter does not show up with ruby -h — "steenslag (siep korteling)" <s.korteling@...>
10 messages
2012/10/18
[#48132] [ruby-trunk - Bug #7201][Open] Setting default_external affects STDIN encoding but default_internal does not — "brixen (Brian Ford)" <brixen@...>
6 messages
2012/10/21
[#48154] Patch to test_ssl to validate server-side support for SNI — Patrick Toomey <ptoomey3@...>
I recently made a pull request to JRuby (
4 messages
2012/10/22
[#48675] Re: Patch to test_ssl to validate server-side support for SNI
— Charles Oliver Nutter <headius@...>
2012/10/31
We have incorporated Patrick's SNI patch for upcoming release JRuby
[#48191] [ANN] 2.0.0 feature freeze — Yusuke Endoh <mame@...>
Japanese later; 日本語は後で
37 messages
2012/10/24
[#48197] Re: [ANN] 2.0.0 feature freeze
— Rodrigo Rosenfeld Rosas <rr.rosas@...>
2012/10/24
Em 24-10-2012 09:39, Yusuke Endoh escreveu:
[#48696] Re: [ANN] 2.0.0 feature freeze
— SASADA Koichi <ko1@...>
2012/11/01
(2012/10/24 5:39), Yusuke Endoh wrote:
[#48701] Re: [ANN] 2.0.0 feature freeze
— Yusuke Endoh <mame@...>
2012/11/01
Hello ko1,
[#48712] Re: [ANN] 2.0.0 feature freeze
— Marc-Andre Lafortune <ruby-core-mailing-list@...>
2012/11/01
Hi,
[#48713] Re: [ANN] 2.0.0 feature freeze
— Yusuke Endoh <mame@...>
2012/11/01
AFAIK matz has not accepted #6636 completely yet.
[#48797] Re: [ANN] 2.0.0 feature freeze
— Clay Trump <clay.trump@...>
2012/11/03
Sorry, late to the party, but what's the status of #6679?
[#48798] Re: [ANN] 2.0.0 feature freeze
— =?KOI8-R?B?4NLJyiDzz8vPzM/X?= <funny.falcon@...>
2012/11/03
What status of #6638 <https://bugs.ruby-lang.org/issues/6638>
[#48260] [ruby-trunk - Bug #7214][Open] Ruby 2.0 breaks support for some debugging tools — "banister (john mair)" <jrmair@...>
22 messages
2012/10/25
[#49657] [ruby-trunk - Bug #7214] Ruby 2.0 breaks support for some debugging tools
— "denofevil (Dennis Ushakov)" <dennis.ushakov@...>
2012/11/20
[#48292] [ruby-trunk - Bug #7216][Open] object.c defines clone method for objects that cannot be cloned. — "therevmj (Michael Johnson)" <mj@...>
6 messages
2012/10/26
[#48315] [ruby-trunk - Bug #7220][Open] StringIO#initialize_copy causes aliasing between the objects — "brixen (Brian Ford)" <brixen@...>
13 messages
2012/10/26
[#48475] [ruby-trunk - Feature #3222] Can bignums have singleton class & methods? — "matz (Yukihiro Matsumoto)" <matz@...>
4 messages
2012/10/27
[#48497] Re: [ruby-trunk - Feature #3222] Can bignums have singleton class & methods?
— SASADA Koichi <ko1@...>
2012/10/27
(2012/10/27 23:25), matz (Yukihiro Matsumoto) wrote:
[#48551] [ruby-trunk - Feature #7241][Open] Enumerable#to_h proposal — "nathan.f77 (Nathan Broadbent)" <nathan.f77@...>
23 messages
2012/10/29
[#48587] [ruby-trunk - Feature #7241] Enumerable#to_h proposal
— "rosenfeld (Rodrigo Rosenfeld Rosas)" <rr.rosas@...>
2012/10/30
[#48625] Re: [ruby-trunk - Feature #7241] Enumerable#to_h proposal
— Aaron Patterson <tenderlove@...>
2012/10/30
On Tue, Oct 30, 2012 at 07:58:33PM +0900, rosenfeld (Rodrigo Rosenfeld Rosas) wrote:
[#48631] Re: [ruby-trunk - Feature #7241] Enumerable#to_h proposal
— Rodrigo Rosenfeld Rosas <rr.rosas@...>
2012/10/30
Em 30-10-2012 16:23, Aaron Patterson escreveu:
[#48679] [ruby-trunk - Feature #905] Add String.new(fixnum) to preallocate large buffer — "headius (Charles Nutter)" <headius@...>
4 messages
2012/10/31
[ruby-core:48308] [ruby-trunk - Bug #7158] require is slow in its bookkeeping; can make Rails startup 2.2x faster
From:
"funny_falcon (Yura Sokolov)" <funny.falcon@...>
Date:
2012-10-26 15:43:53 UTC
List:
ruby-core #48308
Issue #7158 has been updated by funny_falcon (Yura Sokolov).
Checking for "shares same array" is a great catch :) I like it.
But my patch #5767 for $LOAD_PATH so "invasive" cause there is markable benefits from overriding "$:.push", so that whole $LOAD_PATH will not be re-scaped after each $: << gem_path .
Also, which way do you handle changing of current directory? Cache should be invalidated on Dir.chdir .
Also, which way do you handle changing filesystem encoding? One test from 'make check' fails, if you do not invalidate cache on changing filesystem encoding.
load_features_index will be recalculated after each require for new feature also? It is very strange, that you patch for $LOADED_FEATURES gains so much speedup. My patch #5427 gains only 6%, not 40%, and I doubt that hash lookup on ~2000 elements so much faster than crafted binary search (As well as you, I compare only "basename" of a feature). And I have no additional objects: no hashes, no arrays, no substrings, only $LOADED_FEATURES itself sorted in a rignt way.
... but maybe I'm mistacken.
About numbers: speedup 1.40x - is 29% speedup, and 2.20x is 55% speedup. My patches combined gains same 55% speedup as yours.
Can you test with "cache" patch but without "index" patch also?
----------------------------------------
Bug #7158: require is slow in its bookkeeping; can make Rails startup 2.2x faster
https://bugs.ruby-lang.org/issues/7158#change-31613
Author: gregprice (Greg Price)
Status: Assigned
Priority: Normal
Assignee: ko1 (Koichi Sasada)
Category: core
Target version:
ruby -v: ruby 1.9.3p194 (2012-04-20 revision 35409) [i686-linux]
=begin
Starting a large application in Ruby is slow. Most of the startup
time is not spent in the actual work of loading files and running Ruby
code, but in bookkeeping in the 'require' implementation. I've
attached a patch series which makes that bookkeeping much faster.
These patches speed up a large Rails application's startup by 2.2x,
and a pure-'require' benchmark by 3.4x.
These patches fix two ways in which 'require' is slow. Both problems
have been discussed before, but these patches solve the problems with
less code and stricter compatibility than previous patches I've seen.
* Currently we iterate through $LOADED_FEATURES to see if anything
matches the newly required feature. Further, each iteration
iterates in turn through $LOAD_PATH. Xavier Shay spotted this
problem last year and a series of patches were discussed
(in Issue #3924) to add a Hash index alongside $LOADED_FEATURES,
but for 1.9.3 none were merged; Masaya Tarui committed Revision r31875,
which mitigated the problem. This series adds a Hash index,
and keeps it up to date even if the user modifies $LOADED_FEATURES.
This is worth a 40% speedup on one large Rails application,
and 2.3x on a pure-'require' benchmark.
* Currently each 'require' call runs through $LOAD_PATH and calls
rb_file_expand_path() on each element. Yura Sokolov (funny_falcon)
proposed caching this last December in Issue #5767, but it wasn't
merged. This series also caches $LOAD_PATH, and keeps the cache up
to date with a different, less invasive technique. The cache takes
34 lines of code, and is worth an additional 57% speedup in
starting a Rails app and a 46% speedup in pure 'require'.
== Staying Compatible
With both the $LOADED_FEATURES index and the $LOAD_PATH cache,
* we exactly preserve the semantics of the user modifying $LOAD_PATH
or $LOADED_FEATURES;
* both $LOAD_PATH and $LOADED_FEATURES remain ordinary Arrays, with
no singleton methods;
* we make just one semantic change: each element of $LOAD_PATH and
$LOADED_FEATURES is made into a frozen string. This doesn't limit
the flexibility Ruby offers to the programmer in any way; to alter
an element of either array, one simply reassigns it to the new
value. Further, normal path-munging code which only adds and
removes elements shouldn't have to change at all.
These patches use the following technique to keep the cache and the
index up to date without modifying the methods of $LOADED_FEATURES or
$LOAD_PATH: we take advantage of the sharing mechanism in the Array
implementation to detect, in O(1) time, whether either array has been
mutated. We cause $LOADED_FEATURES to be shared with an Array we keep
privately in load.c; if anything modifies it, it will break the
sharing and we will know to rebuild the index. Similarly for
$LOAD_PATH.
== Benchmarks
First, on my company's Rails application, where $LOAD_PATH.size is 207
and $LOADED_FEATURES.size is 2126. I measured the time taken by
'bundle exec rails runner "p 1"'.
. Rails startup time,
version best of 5 speedup
v1_9_3_194 12.197s
v1_9_3_194+index 8.688s 1.40x
v1_9_3_194+index+cache 5.538s 2.20x
And now isolating the performance of 'require', by requiring
16000 empty files.
version time, best of 5 speedup
trunk (at r36920) 10.115s
trunk+index 4.363s 2.32x
trunk+index+cache 2.984s 3.39x
(The timings for the Rails application are based on the latest release
rather than trunk because a number of gems failed to compile against
trunk for me.)
== The Patches
I've attached four patches:
(1) Patch 1 changes no behavior at all. It adds comments and
simplifies a bit of code to help in understanding why patch 3 is
correct. 42 lines, most of them comments.
(2) Patch 2 adds a function to array.c which will help us tell when
$LOAD_PATH or $LOADED_FEATURES has been modified. 17 lines.
(3) Patch 3 adds the $LOADED_FEATURES index. 150 lines.
(4) Patch 4 adds the $LOAD_PATH cache. 34 lines.
Reviews and comments welcome -- I'm sure there's something I could do
to make these patches better. I hope we can get some form of them
into trunk before the next release. My life has been happier since I
switched to this version because commands in my Rails application all
run faster now, and I want every Ruby programmer to be happier in the
same way with 2.0 and ideally with 1.9.4.
=end
--
http://bugs.ruby-lang.org/