From: "byroot (Jean Boussier)" Date: 2022-06-28T12:21:36+00:00 Subject: [ruby-core:109080] [Ruby master Feature#18559] Allocation tracing: Objects created by the parser are attributed to Kernel.require Issue #18559 has been updated by byroot (Jean Boussier). So I tried adding a `frame` before the ISeq is compiled: https://github.com/ruby/ruby/pull/5998 It somewhat works for regular `require` but: - It's awkward to put an uncompiled ISeq in the stack, there is even an `assert` to prevent this. - It doesn't work when loading the ISeq with `load_from_binary` because the source location is inside the ISeq. We could re-order the iseq fields so it's the first loaded element, but still very awkward. So I tried a much simpler, ad hoc solution which is to store an override on the execution context before starting the compilation: https://github.com/ruby/ruby/pull/6057. If present TracePoint expose that. It's not very clean conceptually, but seem much simpler and safer. @ko1 expressed some reservations about it, but I don't see any other solution. Opinions welcome. ---------------------------------------- Feature #18559: Allocation tracing: Objects created by the parser are attributed to Kernel.require https://bugs.ruby-lang.org/issues/18559#change-98224 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- Marking this as a feature, because I think it should be improved but can hardly be considered a bug. ### Repro Consider the following script: ```ruby # /tmp/allocation-source.rb require 'objspace' require 'tmpdir' source = File.join(Dir.tmpdir, "foo.rb") File.write(source, <<~RUBY) # frozen_string_literal: true class Foo def plop "fizz" end end RUBY ObjectSpace.trace_object_allocations_start GC.start gen = GC.count require(source) ObjectSpace.dump_all(output: $stdout, since: gen) ``` ### Expected behavior I'd expect the `ObjectSpace.dump_all` output to attribute all new objects, including `T_IMEMO` etc, to `foo.rb` ### Actual behavior They are attributed to the source file that called `Kernel.require` (so with `--disable-gems`): ``` {"address":"0x11acaec78", "type":"CLASS", "class":"0x11acaebb0", "superclass":"0x10fa4a848", "name":"Foo", "references":["0x10fa4a848", "0x11acaea98", "0x11acaf790"], "file":"/var/folders/vy/srfpq1vn6hv5r6bzkvcw13y80000gn/T/foo.rb", "line":2, "generation":1, "memsize":544, "flags":{"wb_protected":true}} {"address":"0x11acaeca0", "type":"IMEMO", "class":"0x8", "imemo_type":"cref", "references":["0x10fa4a848"], "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":40, "flags":{"wb_protected":true}} {"address":"0x11acaecc8", "type":"STRING", "class":"0x10fa42418", "frozen":true, "embedded":true, "fstring":true, "bytesize":4, "value":"fizz", "encoding":"UTF-8", "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":40, "flags":{"wb_protected":true}} {"address":"0x11acaecf0", "type":"ARRAY", "class":"0x10fa28f68", "frozen":true, "length":2, "embedded":true, "references":["0x11acaff88", "0x11acaf240"], "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":40, "flags":{"wb_protected":true}} {"address":"0x11acaed18", "type":"IMEMO", "imemo_type":"iseq", "references":["0x11acaecc8", "0x11acaf600", "0x11acaf600", "0x11acaecf0"], "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":416, "flags":{"wb_protected":true}} {"address":"0x11acaf1a0", "type":"ARRAY", "class":"0x10fa28f68", "frozen":true, "length":2, "embedded":true, "references":["0x11acaff88", "0x11acaf240"], "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":40, "flags":{"wb_protected":true}} {"address":"0x11acaf1c8", "type":"IMEMO", "imemo_type":"iseq", "references":["0x11acaed18", "0x11acaf1f0", "0x11acaf1f0", "0x11acaf1a0", "0x11acaf290"], "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":456, "flags":{"wb_protected":true}} {"address":"0x11acaf1f0", "type":"STRING", "class":"0x10fa42418", "frozen":true, "embedded":true, "fstring":true, "bytesize":11, "value":"", "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":40, "flags":{"wb_protected":true}} {"address":"0x11acaf218", "type":"ARRAY", "class":"0x10fa28f68", "frozen":true, "length":2, "embedded":true, "references":["0x11acaff88", "0x11acaf240"], "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":40, "flags":{"wb_protected":true}} {"address":"0x11acaf240", "type":"STRING", "class":"0x10fa42418", "frozen":true, "fstring":true, "bytesize":63, "value":"/private/var/folders/vy/srfpq1vn6hv5r6bzkvcw13y80000gn/T/foo.rb", "encoding":"UTF-8", "file":"/tmp/allocation-source.rb", "line":19, "method":"require", "generation":1, "memsize":104, "flags":{"wb_protected":true}} .... ``` ### Why is it a problem? This behavior makes it impossible to properly analyze which part of an application use the most memory. For instance when using `heap-profiler` on an app using `Bootsnap`, all objects created as a result of loading source file are attributed to bootsnap: ``` retained memory by gem ----------------------------------- 351.64 MB bootsnap-1.10.2 ``` If this behaved as I expect, `heap-profiler` would be able to report how much each gem contribute to the app RAM usage. -- https://bugs.ruby-lang.org/ Unsubscribe: