Make Ruby Differentiable

Make Ruby Differentiable

• 1.
  Make Ruby Differentiable (AutomaticDifferentiation for Ruby) @nagachika 2019-04-19 RubyKaigi 2019
• 2.
  Make Ruby Differentiable ThisLT is homage for the episode #206 of Rebuild.fm
• 3.
  Make Ruby Differentiable Thereare two LT talks for ‘Automatic Differentiation’!!
• 4.
  Swift for TensorFlow https://github.com/tensorflow/swift/blob/master/docs/AutomaticDifferentiation.md
• 5.
  Automatic Differentiation Berland atEnglish Wikipedia [Public domain],
 via Wikimedia Commons
• 6.
  Differentiation Gem https://rubygems.org/gems/differentiation https://github.com/nagachika/differentiation
• 7.
  differential def func(x,y) (x + 1.0) * (y - 2.0) end result = func(1.0, 2.0) result => 0.0 result.gradients => { x: 0.0, y: 2.0 } result.gradients(:x, :y) => [ 0.0, 2.0 ] Make method differentiable
• 8.
  l = ->(x,y){ (x + 1.0) * (y - 2.0) } differentiable_l = differential l result = differentiable_l.call(1.0, 2.0) result => 0.0 result.gradients => { x: 0.0, y: 2.0 } result.gradients(:x, :y) => [ 0.0, 2.0 ] Make Proc differentiable
• 9.
  x = 1.0.to_dual_number y= 2.0.to_dual_number result = (x + 1.0) * (y - 2.0) result => 0.0 result.gradients(x, y) => [ 0.0, 2.0 ] Make Numeric Differentiable
• 10.
  mat = Matrix[ [0.0, 1.0], [ 2.0, 3.0] ] differential def matmul_reduce(m) (m * m).sum end result = matmul_reduce(mat) => 22.0 result.gradients(:m) => [Matrix[[3.0, 7.0], [5.0, 9.0]]] Make Matrix Differentiable
• 11.
  class DualNumber … def coerce(other) ifDifferentiation.differentiable?(other) [ Differentiation.convert_to_dual_number(other), self] else super end end end Numeric#coerce
• 12.
  Differentiable DEMO Multiple LayeredPerceptron solving XOR x1 x2 y 0 0 0 0 1 1 1 0 1 1 1 0 x1 x2 y
• 13.
  Differentiable DEMO Multiple LayeredPerceptron solving XOR
 with Matrix and differentiation.gem
• 14.
  Let’s Make RubyDifferentiable