Download as ODP, PPTX
Uploaded byLourens Naudé
RailswayCon 2010 - Dynamic Language VMs
The document discusses Ruby virtual machines and optimizations. It covers topics like Ruby VM internals, system resources, Ruby optimizations like constant folding and inlining, object structures, classes and modules, method dispatch, garbage collection. The goal is to understand tools and techniques for building high performance Ruby applications.
More Related Content
![[ASM]Lab6](https://cdn.slidesharecdn.com/ss_thumbnails/asmlab6-151121102137-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[ASM]Lab4](https://cdn.slidesharecdn.com/ss_thumbnails/asmlab4-151121101809-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to RailswayCon 2010 - Dynamic Language VMs
![Governance, Deployment & Methodologies for Agentic Automation [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day2-251112135038-a386476d-thumbnail.jpg?width=640&height=640&fit=bounds)
RailswayCon 2010 - Dynamic Language VMs
- 1. Dynamic Language VMsRuby 1.9 Lourens Naude, WildfireApp.com
- 2. Background Independent ContractorRuby / C / integrations
- 3. Well versed fullstack
- 4. Architecture WildfireApp.com SocialMarketing platform
- 5.
- 6. Bursty traffic –Lady Gaga, EA, Gatorade etc.
- 7.
- 8.
- 9. A GOOD CRAFTSMENKNOWS HIS TOOLS
- 10. A BAD CRAFTSMENBLAMES HIS TOOLS
- 11. Typical public facingapps Interaction patterns Request / response
- 12.
- 13. Event driven OverheadsData transfer (I/0)
- 14.
- 15. VM – allocation,symbol tables etc. (CPU + mem)
- 16.
- 17. Ruby daemon -strace Process 5856 detached % time calls syscall ------ ------- ------------- 89.69 5092 recvfrom 5.35 5093 sendto 2.49 26300 stat 2.05 11004 clock_gettime
- 18. Ruby daemon -ltrace % time calls function ------ -------- -------- 95.78 635173 memcpy 1.38 25862 malloc 0.79 14984 free 0.60 11403 strcmp
- 19. System Resources Datalatency CPU cache
- 20.
- 21.
- 22. Memory + disk- remote Record retrieval with ORM Fetch results (local/remote memory + disk)
- 23.
- 24. Object instantiation (CPU+ memory)
- 25. Optional memcached (localor remote memory)
- 26.
- 27. Conversion – rowsto hash Benchmark.bm do |b| b.report do 1000.times{ ActiveRecord::Base.connection.select_rows "SELECT * FROM users" } end end user system total real 0.300000 0.040000 0.340000 ( 0.505095)
- 28. Conversion – rowsto objects Benchmark.bm do |b| b.report do 1000.times{ ActiveRecord::Base.connection.select_all "SELECT * FROM users" } end end user system total real 0.510000 0.050000 0.560000 ( 0.719201)
- 29. Instantiation Benchmark.bm do|b| b.report do 100_000.times{ 'string'.dup } end end user system total real 0.040000 0.000000 0.040000 ( 0.043791)
- 30. Serialization – load+ dump Benchmark.bm do |b| b.report do 100_000.times{ Marshal.load(Marshal.dump('ruby string')) } end end user system total real 1.660000 0.010000 1.670000 ( 1.699882)
- 31. Roadmap VM ArchitectureSymbol table
- 32.
- 33.
- 34. Optimizations Ruby languageObject model
- 35.
- 36.
- 37.
- 38.
- 39.
- 40. Changes Ruby 1.8artifacts Parser && AST nodes
- 41.
- 42.
- 43. No immediate performancegains for String manipulation etc. Codegen phase Better optimization hooks
- 44.
- 45.
- 46.
- 47. Abstract Syntax Tree(AST) Structure Grammar representation
- 48. Annotations attach semanticsto nodes
- 49. Possible to refactorthe tree – more nodes, less complexity Example nodes Literals, values and assignments
- 50. Method calls, argumentsand return values
- 51. Jumps – if,else, iterators
- 52. Unconditional jumps –exceptions, retry etc.
- 53. Code generation Howit works Converts the AST to compiled code segments
- 54. Reduces a treeto a linear and ordered instruction set
- 55. Fast execution –no tree walking + native code Workflow Preprocessing – AST refactoring (!YARV)
- 56. Codegen, nodes ->instruction sequences
- 57. Postprocessing – replacewith optimal instruction sequences (peephole optimization)
- 58. Pre and postprocessingphases may be multiple passes
- 59.
- 60.
- 61. Symbol / Hashtables How it works Constant time access to int/char indexed values
- 62. Table defaults: 11bins, 5 entries per bin
- 63. Bins++, sequential lookupinside bins
- 64. Lookup of methods,variables, encodings etc. Symbol Entity with both a String and Number representation
- 65. !(String || Symbol),points to a table entry
- 66. Developer identifies byname, VM by int
- 67. Immutable for performance– watch out for memory
- 68.
- 69. VM instructions /opcodes Stateless functions 80+ currently
- 70. Generated from definitionsat interpreter compile time (existing ruby requirement for 1.9)
- 71. Instruction / opcode/ operands notation Categories and examples variable: get or set local variable
- 72. class / module:definition
- 73. method / iterator:invoke method, call block
- 74. Optimization: redefines common+, <<, * contracts
- 75. Managing opcode sequencesStack Machine 2 instruction types: push && pop
- 76. Move / copyvalues, top of stack -> elsewhere
- 77. SP: top ofstack pointer, BP: bottom of stack pointer Example %w(a b c)
- 78. Put strings “a”,“b” and “c” on the stack
- 79. Fetch top 3stack elements
- 80. Create an arrayfrom them
- 81. Instruction sequence Opcodecollection Instruction dispatch can be a bottleneck
- 82. Optimizing simple instructionsis very important
- 83. Likely a smallsubset of the typical web app's hot path Dispatch techniques Direct Threaded Dispatch : fastest jump to next opcode / instruction
- 84. Switch Dispatch :slower, but portable
- 85.
- 86. Dispatch techniques DirectThreaded Dispatch Represents an instruction by the address of the routine that implements it
- 87.
- 88. Not portable: GCCfirst class labels Switch Dispatch CPU branch mispredictions, depending on pipeline length
- 89. Up to 50%slower than Threaded dispatch
- 90.
- 91. VM Caches VersioningState counter scopes caches to the current VM state
- 92. Lazy invalidation –just bump the version Expires on constant definition
- 93.
- 94.
- 95.
- 96. method cache changes(covered later)
- 97.
- 98. Optimization Limitations StaticAnalysis Examine source code without execution
- 99. Dynamic analysis –runtime introspection Dynamic nature of Ruby Literals are generally safe to consider for optimizations
- 100. Constants can beredefined
- 101. Open classes –variable method table
- 102.
- 103. No explicit returntypes
- 104.
- 105.
- 106.
- 107.
- 108.
- 109.
- 110.
- 111. * not allimplemented in YARV
- 112. Constant folding 1+ 2 # 3 2 * 3 # 3 + 3
- 113. 2 * 1 # 2
- 114. 2 ** 2# 2 *2
- 115.
- 116. def +(*args) #dynamic Ruby spec
- 117.
- 118.
- 119. Code elimination loop{ # loop { begin # begin # eval'ed code # eval'ed code break # break break # ensure ensure # end end # } }
- 120. Subexpression elimination x= x – (y * 2) z = z – (y * 2) t = y * 2 x = x – t z = z - t
- 121. Constant propagation defa b = 20 c(3 * b) end def a # def a b = 20 # c(60) c(3 * 20) # end end
- 122. In-lining def b2 * 3 end def a # def a def a 2 + b # 2 + 2 * 3 2 + (2 * 3) end # end end
- 123. Cloning def a(b,c) b << c expire_cache end a('railsway', 'con') def a_railsway_con 'railsway' << 'con' expire_cache end
- 124. Peephole Optimization (before)x = true # 0008 getlocal x if x # 0010 branchunless 17 else # 0012 jump 14 end # 0014 putnil 0015 jump 18 0017 putnil 0018 leave
- 125. Peephole Optimization (after)x = true # 0008 getlocal x if x # 0010 branchunless 15 else # 0012 putnil end # 0013 leave 0014 pop 0015 putnil 0016 leave
- 126.
- 127.
- 128. Identity Unique identifierto represent the object at runtime Methods Change or query object state
- 129. Command and Querypattern
- 130. Object structure typedefunsigned long VALUE; struct RBasic { VALUE flags; # object flags VALUE klass; # instance of ... }
- 131. Object structure (cont)Casting Pointer type that represent addresses to language structures
- 132.
- 133. ((struct RBasic *)obj)->flagsFlags frozen
- 134.
- 135.
- 136.
- 137. Classes / modulesstructure struct RClass { struct RBasic basic; # object structure rb_classext_t *ptr; # external class struct st_table *m_tbl; # method table struct st_table *iv_index_tbl; # ivars }
- 138. Class / modulestructure (cont) Casting RCLASS(a_str)->ptr.super #=> Object
- 139. RCLASS(a_fixnum)->ptr.super #=> IntegerAttributes Symbol tables for methods and ivars
- 140. Class / moduledistinction through flags
- 141. Special objects ImmediatesNo runtime casting overheads – fits in VALUE
- 142.
- 143.
- 144.
- 145.
- 146. Fixnums <= 30bits
- 147. Floats and Bignumare complex objects – hence poor Floating Point benchmarks
- 148.
- 149. Object memory layoutObject#object_id (32 bit architecture) sizeof(VALUE) is 4 bytes
- 150.
- 151.
- 152.
- 153.
- 154. Mutable Objects structRString { struct RBasic basic; union {struct {long len; char *ptr union { long capa; VALUE shared; }aux; }heap;
- 155. Mutable Objects (cont)String and Array require the ability to shrink / grow capacity
- 156. allocates slightly moredata than required
- 157. Avoids malloc, reallocand memmove overhead
- 158.
- 159.
- 160. Shared Objects str= 'railsway'; str2 = “#{str}con” # shared ref str3 = str << 'con' # copy + mod ary = %w(railsway con) ary2 = ary.dup # shared ref ary3 = ary2.delete_at(1) # copy + mod
- 161. Method Dispatch Languageconstraints Loose typing
- 162.
- 163. Method calls cannever be reduced to CALL(a_method)
- 164. Search overhead Languageconstraints
- 165.
- 166.
- 167.
- 168. Call method ordelegate to superclass
- 169.
- 170. call VS sendobj.__send__ :method We never call methods
- 171. Send query /command messages to objects
- 172. Methods return values– RPC style messaging Method cache Method cache == router
- 173. 95% hit ratewhen warm
- 174. Method redefinition, moduleinclusion etc. clears the method cache / “routing table”
- 175. Introduces significant overheadfor subsequent method calls
- 176. Method cache don'tsclass SomeController < AC::Base def show
- 177. # busts methodcache for the whole VM
- 178.
- 179.
- 180.
- 181.
- 182.
- 183.
- 184. Instance var changesOptimizations First 3 ivars is embedded on the object
- 185. Avoids symbol tablelookups ivar table Table is per class, not per object
- 186. Ivar table isshared by all instances of the same class
- 187. Saves on memoryfootprint of a table per instance
- 188.
- 189. Process memory layoutCode segment Executable code
- 190. Read only Stacksegment Stack storage
- 191. Addressed with stackpointers Heap Memory available for program / developer use
- 192. Malloc Usable /free space Managed by a free list
- 193. Linear search overheadto find free chunks Better layout Index free chunks by size intervals
- 194.
- 195.
- 196. GC terminology Rootset Directly accessible without pointer scanning
- 197. C stack, globalvars, global constants etc. Unreachable hooks Variable assignment to nil
- 198. method return etc.Conservative VM hands out raw pointers to objects
- 199. GC strategies Stopthe World Minimal allocation overhead
- 200. Hands out objectswhile heap space is available
- 201. Halts execution toreclaim memory
- 202. Very disruptive inthe hot path Incremental Collection activity during allocation
- 203. Smoother, but withsome minor overhead
- 204. Suitable for hardrealtime environments
- 205. Scripting GC Markand Sweep Identifies live objects
- 206. Assumes remainder isfor collection
- 207. Concerned with unreachableobjects Stop and Copy 2 heap spaces (double memory overhead)
- 208. 1 active, 1inactive
- 209. Copies reachable chunksto the new active area
- 210. Concerned with liveobjects
- 211. Common GC IssuesConservative GC Memory fragmentation
- 212.
- 213. Memory leaks fromcircular garbage Allocation Bursty allocation
- 214. Knowledge of pointerlayout and chunks required
- 215. Ruby heap layoutMultiple heaps Referenced through heap list
- 216. Composed of multipleslots
- 217.
- 218. IF all slotsis tagged as being free
- 219. A Rails appallocates 4 to 6 heaps on startup
- 220.
- 221.
- 222.
- 223. Slot layouts Perheap Each slot references a single object
- 224. Defaults to 10000 slots for the first heap
- 225. Threshold of 4096free slots per heap
- 226. Free list pointsto the next free slot Heap growth Next allocated heap has 1.8 capacity of the last one
- 227. That's why memoryconsumption's so high ...
- 228. Heap growth –small app >> 8 * 1.8 => 14.4
- 229. >> 8 *1.8 * 1.8
- 230.
- 231. >> 8 *1.8 * 1.8 * 1.8
- 232.
- 233. >> 8 *1.8 * 1.8 * 1.8 * 1.8
- 234.
- 235. Heap growth –mid to large app => 83.9808 >> 8 * 1.8 * 1.8 * 1.8 * 1.8 * 1.8
- 236.
- 237. >> 8 *1.8 * 1.8 * 1.8 * 1.8 * 1.8 * 1.8
- 238.
- 239. >> 8 *1.8 * 1.8 * 1.8 * 1.8 * 1.8 * 1.8 * 1.8
- 240.
- 241. Slot structure typedefstruct RVALUE { union {
- 242.
- 243. VALUE flags; /*0 when free */
- 244.
- 245.
- 246.
- 247.
- 248.
- 249. Pointer layout Selfdescribing Program data area and heap
- 250. RVALUE union canaccommodate any ruby object
- 251. Frames, variable structuresetc. well defined also
- 252. 40 bytes (64bit arch) represents a slot
- 253. Free list pointsto the next free slot
- 254. Ruby heap VSOS heap Ruby heap 20 bytes represents a slot
- 255. slot points toOS data, on the OS / system heap OS heap Thus a 20 byte slot can reference a 2MB chunk on the system heap
- 256.
- 257. CRuby: Mark andSweep Conservative Cannot determine with certainty if a value references an object – assume it's in use Two phase implementation Mark phase: identifies and flags reachable objects from the current program context
- 258. Sweep phase: iteratesthrough the object space and …
- 259. free all objectsnot marked
- 260.
- 261.
- 262. Work proportional toheap size
- 263. Prone to memoryfragmentation (no compaction)
- 264. Recursive Triggers 8mmalloc calls triggers GC
- 265. Every 8MB allocatedtriggers GC
- 266. Not enough heapreserve
- 267. GC in action# 4 objs, 1 Array, 3 Strings ary1 = %w(a b c)
- 268. ary2 = %w(de f)
- 269. # both ary1and ary2 is reachable
- 270.
- 271. # ary1 andit's contents is unreachable
- 272.
- 273.
- 274.
- 275. Generational GC ObservationsVast majority of objects are short lived – 80%+
- 276. Expensive to accountfor long lived objects
- 277. Parition by ageand frequently collect short lived ones How it works Restrict GC to the most recently modified slots
- 278. These “sub heaps”are referred to as generations
- 279. Perform a fullGC only when the youngest generation
- 280. fails to meetmemory requirements
- 281.
- 282. Threading Changes NativeOS Threads
- 283. Ruby Thread ==pthread
- 284. Multiple cores ftw!… but Syscalls schedule, synchronize and create
- 285. Much more expensiveto spawn and switch than green threads
- 286. Global VM Lock(GVL)
- 287. Global VM Lock(GVL) How it works Thread that owns the GVL is allowed to execute
- 288. Blocking operations shouldrelease the GVL
- 289. Automatically released whenscheduled
- 290. C extensions :author does not concern with syncronization
- 291. Blocking VM operationsI/O blocking reads and writes
- 292. DNS resolution orconnects
- 293. Often has hugehandshake overheads Computations, processes and locks Expensive Bignum ops blocked 1.8 interpreters
- 294.
- 295.
- 296. Releasing the GVLStable API Blocking function: slow system call / computation
- 297. Unblock function: calledon Thread interrupt Pitfalls
- 298. Cannot access VALUEs(objects) in blocking functions
- 299. No integration withRuby's exception / error handler
- 300. Lightweight Concurrency FibersCoroutines – 4k stack size
- 301. Very fast userspace context switches
- 302.
- 303. Fiber.yield pauses theactivation record, which keeps context across multiple calls Use cases Generators
- 304. Blocking I/0 -Neverblock
- 305. In the pipelineMVM: Multiple Virtual Machines Shared process state
- 306. Sandboxed per VMapplication state
- 307. Distribute VMs acrossavailable cores
- 308. Message passing forinter VM communication
- 309. Most Ruby deploymentsaren't thread safe
- 310. MVM is wellsuited for this
- 311.