Ruby/RubyGems/Bundler
Hiroshi SHIBATA / GMO Pepabo, Inc.
2018.10.25 rubyconf.my 2018
Gems on Ruby
self.introduce
self.introduce
=> {
name: “SHIBATA Hiroshi”,
nickname: “hsbt”,
organizations: [“ruby”, “rubygems”, “bundler”,
“asakusarb”, “railsgirls”, “pepabo”, …],
commit_bits: [“ruby”, “rake”, “rubygems”, “bundler”,
“rdoc”, “psych”, “json”, “ruby-build”, “railsgirls”,
“railsgirls-jp”, …],
sites: [“hsbt.org”, “ruby-lang.org”, “rubyci.org”,
“railsgirls.com”, “railsgirls.jp”],
}
• Ruby Core Team
• Ruby 2.6+
• RubyGems
• RubyGems 3.0/4.0
• Bundler
• RubyGems&Bundler
Agenda
Ruby Core team
1.
What’s Ruby Core Team?
Heroku
• matz
• n0kada
$ cat ~svn/.ssh/authorized_keys | awk '{print $5}' | sort | uniq | wc -l
99
Total 97 people + 2 bots
Money Forward
• shyouhei
Full-time commiters
Cookpad
• ko1
• mametter
Speee
• mrkn
Branch maintainers
trunk known as 2.6
@nurse: Release manager
2.5
@nagachika: Stable branch maintainer
2.4 and 2.3
@unak: Old stable branch maintainer
Sponsors of the Ruby language
• heroku:
Provide the free Enterprise account
• fastly:
OSS plan of CDN for *.ruby-lang.org
• NaCl:
Network and Compute resources for website
• Ruby Association:
Grant of development for cloud resources
• Nihon Ruby no Kai:
Grant of development for hardware
Ruby 2.6+
2.
Version number and release cycle
We plan to release every Christmas day.
• 2.1.0: 2013/12/25
• 2.2.0: 2014/12/25
• 2.3.0: 2015/12/25
• 2.4.0: 2016/12/25
• 2.5.0: 2017/12/25
• 2.6.0: 2018/12/25(TBD)
• …
• 3.0.0: 2020/xx/xx
2/24: 2.6.0preview1 release
It contained MJIT feature
see https://bugs.ruby-lang.org/issues/14235
5/31: 2.6.0preview2 release
RubyGems 3.0.0.beta1, Improve MJIT Performance
11/?: 2.6.0preview3 release
RubyGems 3.0.0.beta2, Psych-3.1.0.pre2
12/25: 2.6.0 final release
Ruby 2.6 timeframe
•JIT(Just In Time) Comile feature for Ruby
•It’s developed by @k0kubun
•See https://medium.com/@k0kubun/the-method-jit-compiler-
for-ruby-2-6-388ee0989c13
What’s MJIT?
• Standard Libraries
• Upstream: Only Ruby core repository(svn.ruby-lang.org)
• Release cycle: 1 year
• Default Gems
• Upstream: Ruby core repository and GitHub
• Release cycle: 1year or maintainers matter
• Bundled Gems
• Upstream: Only GitHub
• Release cycle: Maintainers matter
Gemification Project
Classification of standard libraries
In Ruby 2.5
Standard Libraries
• Pure ruby: 64
• Extensions: 14
Default gems
• Pure ruby: 6
• Extensions: 14
Bundled Gems
• Pure ruby: 7
• Extensions: 0
In Ruby 2.6
Standard Libraries
• Pure ruby: 49(-15)
• Extensions: 14
Default gems
• Pure ruby: 21 (+15)
• Extensions: 14
Bundled Gems
• Pure ruby: 7
• Extensions: 0
git.ruby-lang.org
RubyGems
3.
• The package manager of Ruby libraries.
• `gem install rails -v “~> 5.2”`
• You can install specified version of Ruby libraries
that called `Gem`. RubyGems handles global
environment on your box.
• You could specify `gem ‘rails’, ‘~> 5.2’` syntax
without its dependency.
What’s rubygems?
What does mean “official”?
“official” means “Matz controllable”
Un-controllable examples:
• ruby-doc.org (not docs.ruby-lang.org)
• rubygems.org
• bundler.io
• Ruby version manager(rvm/rbenv/chruby)
• The RubyGems accepts SemVer Policy.
• Merge latest stable version into Ruby Core
• Ruby 2.6.0 will bundle RubyGems 3.0(TBD)
• Ruby 2.7 or 3.0 will bundle RubyGems 4.0(TBD)
The policy of RubyGems versioning
• RubyGems also have HackerOne.
• 3 people handle vulnerability issues and will
release RubyGems by SemVer Policy like “2.7.7”
from “2.7.6”
• On the other hand, The Ruby core team will back
port only vulnerability fixes by independent
version like “2.6.5.1”, not “2.7.7”
Security release of RubyGems
RubyGems 3 & 4
4.
• Removed deprecated methods.
• Removed to support for < Ruby 2.2.
• Added warnings of deprecated methods.
• Removed deprecated options.
• [CAUTION] `--ri` and `--rdoc` options
What’s new in RubyGems3?
• Surprisedly, RG 2.7 still supports Ruby 1.8.
Ruby 1.8 in 2018
~/D/g/r/rubygems (2.7) > rg respond_to
test/rubygems/test_gem_request_set_gem_dependency_api.rb
630: tf.close! if tf.respond_to? :close!
test/rubygems/test_gem_source.rb
60: response.uri = URI('http://example') if response.respond_to? :uri
test/rubygems/test_gem_package.rb
755: tf.close! if tf.respond_to? :close!
test/rubygems/test_gem_util.rb
45: if File.respond_to?(:realpath)
test/rubygems/test_gem_installer.rb
58: str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
65:if Gem.respond_to?(:activate_bin_path)
893: skip unless "".respond_to?(:force_encoding)
test/rubygems/test_gem_specification.rb
2305: s.required_rubygems_version = Gem::Requirement.new("> 0".freeze) if s.respond_to? :required_ruby
2316: if s.respond_to? :specification_version then
…snip
• We can use Keywords argument, Refinement,
Other cool features in RubyGems now.
• Simple build matrix
Only support Ruby 2.2+
• It has non-compatible features.
• Make enable as default for conservative option:
https://github.com/rubygems/rubygems/pull/2233
• Make ruby gem install to user-install by default:
https://github.com/rubygems/rubygems/issues/1394
• Executables in bin folder conflict with their gem
versions: https://bugs.ruby-lang.org/issues/5060
• Behaviour changes with default gems installer: https://
github.com/rubygems/rubygems/pull/2166
What’s new in RubyGems4?
• We got the installation time when already installed
gems.
• To use conservative is ignore re-install action.
Make conservative option as default
~ > gem i rails
clone http://rubyonrails.org -> /Users/hsbt/Documents/rubyonrails.org
git ls-remote http://rubyonrails.org
hg identify http://rubyonrails.org
svn info http://rubyonrails.org
error Could not find version control system: http://rubyonrails.org
exists /Users/hsbt/Documents/github.com/rails/rails
Successfully installed rails-5.2.0
1 gem installed
~ > gem i rails —conservative
~ >
• Rubygems 4 will install the all gems to `~/.gem`
• Pros: Ruby in linux distribution has many of FAQ for gem
installation for using `sudo`. This change resolve this
issues.
• Cons: Ruby version manager like rbenv is not support it.
And This is big incompatible feature.
Make `--user-install` as default
Bundler
5.
• The vendoring tool of Ruby.
• RubyGems couldn’t care dependency of Ruby
libraries and isolate version managing with ruby
process.
• Bundler can do them with `Gemfile`
What’s bundler?
# frozen_string_literal: true
source "https://rubygems.org"
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec
# We need a newish Rake since Active Job sets its test tasks' descriptions.
gem "rake", ">= 11.1"
• Maybe, It’s this year.
• We disabled the incompatible features like
renaming `gems.rb` from `Gemfile`
• We will release 1.17.0 at this week. After that, We
bump its version to 2.0.0 and drop to support
under the Ruby 2.2.
When is coming Bundler 2?
RubyGems
&
Bundler
6.
• We are working to integrate RubyGems and
Bundler.
• RubyGems 3&4 drop to support under the Ruby
2.2. Because Bundler 1.x still supports Ruby 1.8
and 1.9.
• I’m waiting to release Bundler 2 for this
integration. After that, I will promote Bundler 2 as
standard library of the Ruby Core.
RubyGems/Bundler integration
• Bundler was located rubygems repository as git
submodule
Bundler Integration(rubygems.rb)
if USE_BUNDLER_FOR_GEMDEPS
ENV["BUNDLE_GEMFILE"] ||= File.expand_path(path)
require 'rubygems/user_interaction'
Gem::DefaultUserInteraction.use_ui(ui) do
require "bundler"
@gemdeps = Bundler.setup
Bundler.ui = nil
@gemdeps.requested_specs.map(&:to_spec).sort_by(&:name)
end
else
rs = Gem::RequestSet.new
@gemdeps = rs.load_gemdeps path
rs.resolve_current.map do |s|
s.full_spec.tap(&:activate)
end
end
• RubyGems 2.x, 3.x uses Molinillo-0.5.7
• Bundler 1.16.x also uses Molinillo-0.6.4
• These are different versions and behavior of
dependency resolver.
Dependency Resolver incompatible
~/D/g/r/rubygems (master) > ls lib/rubygems/resolver/molinillo/lib/molinillo
delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb
dependency_graph errors.rb modules resolver.rb
~/D/g/b/bundler (master) > ls lib/bundler/vendor/molinillo/lib/molinillo
compatibility.rb dependency_graph errors.rb modules resolver.rb
delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb
❤
Executive Officer CPO(Chief Productivity Officer)
Director of Business Process Re-engineering Office
Director of Technical Division
at GMO Pepabo, Inc. @pepabo
Hiroshi SHIBATA @hsbt
https://www.hsbt.org
Ruby is designed to make
programmers happy.
Yukihiro Matz Matsumoto

Gems on Ruby

  • 1.
    Ruby/RubyGems/Bundler Hiroshi SHIBATA /GMO Pepabo, Inc. 2018.10.25 rubyconf.my 2018 Gems on Ruby
  • 2.
  • 3.
    self.introduce => { name: “SHIBATAHiroshi”, nickname: “hsbt”, organizations: [“ruby”, “rubygems”, “bundler”, “asakusarb”, “railsgirls”, “pepabo”, …], commit_bits: [“ruby”, “rake”, “rubygems”, “bundler”, “rdoc”, “psych”, “json”, “ruby-build”, “railsgirls”, “railsgirls-jp”, …], sites: [“hsbt.org”, “ruby-lang.org”, “rubyci.org”, “railsgirls.com”, “railsgirls.jp”], }
  • 5.
    • Ruby CoreTeam • Ruby 2.6+ • RubyGems • RubyGems 3.0/4.0 • Bundler • RubyGems&Bundler Agenda
  • 6.
  • 7.
    What’s Ruby CoreTeam? Heroku • matz • n0kada $ cat ~svn/.ssh/authorized_keys | awk '{print $5}' | sort | uniq | wc -l 99 Total 97 people + 2 bots Money Forward • shyouhei Full-time commiters Cookpad • ko1 • mametter Speee • mrkn
  • 9.
    Branch maintainers trunk knownas 2.6 @nurse: Release manager 2.5 @nagachika: Stable branch maintainer 2.4 and 2.3 @unak: Old stable branch maintainer
  • 10.
    Sponsors of theRuby language • heroku: Provide the free Enterprise account • fastly: OSS plan of CDN for *.ruby-lang.org • NaCl: Network and Compute resources for website • Ruby Association: Grant of development for cloud resources • Nihon Ruby no Kai: Grant of development for hardware
  • 11.
  • 12.
    Version number andrelease cycle We plan to release every Christmas day. • 2.1.0: 2013/12/25 • 2.2.0: 2014/12/25 • 2.3.0: 2015/12/25 • 2.4.0: 2016/12/25 • 2.5.0: 2017/12/25 • 2.6.0: 2018/12/25(TBD) • … • 3.0.0: 2020/xx/xx
  • 13.
    2/24: 2.6.0preview1 release Itcontained MJIT feature see https://bugs.ruby-lang.org/issues/14235 5/31: 2.6.0preview2 release RubyGems 3.0.0.beta1, Improve MJIT Performance 11/?: 2.6.0preview3 release RubyGems 3.0.0.beta2, Psych-3.1.0.pre2 12/25: 2.6.0 final release Ruby 2.6 timeframe
  • 14.
    •JIT(Just In Time)Comile feature for Ruby •It’s developed by @k0kubun •See https://medium.com/@k0kubun/the-method-jit-compiler- for-ruby-2-6-388ee0989c13 What’s MJIT?
  • 15.
    • Standard Libraries •Upstream: Only Ruby core repository(svn.ruby-lang.org) • Release cycle: 1 year • Default Gems • Upstream: Ruby core repository and GitHub • Release cycle: 1year or maintainers matter • Bundled Gems • Upstream: Only GitHub • Release cycle: Maintainers matter Gemification Project
  • 16.
    Classification of standardlibraries In Ruby 2.5 Standard Libraries • Pure ruby: 64 • Extensions: 14 Default gems • Pure ruby: 6 • Extensions: 14 Bundled Gems • Pure ruby: 7 • Extensions: 0 In Ruby 2.6 Standard Libraries • Pure ruby: 49(-15) • Extensions: 14 Default gems • Pure ruby: 21 (+15) • Extensions: 14 Bundled Gems • Pure ruby: 7 • Extensions: 0
  • 17.
  • 19.
  • 20.
    • The packagemanager of Ruby libraries. • `gem install rails -v “~> 5.2”` • You can install specified version of Ruby libraries that called `Gem`. RubyGems handles global environment on your box. • You could specify `gem ‘rails’, ‘~> 5.2’` syntax without its dependency. What’s rubygems?
  • 22.
    What does mean“official”? “official” means “Matz controllable” Un-controllable examples: • ruby-doc.org (not docs.ruby-lang.org) • rubygems.org • bundler.io • Ruby version manager(rvm/rbenv/chruby)
  • 23.
    • The RubyGemsaccepts SemVer Policy. • Merge latest stable version into Ruby Core • Ruby 2.6.0 will bundle RubyGems 3.0(TBD) • Ruby 2.7 or 3.0 will bundle RubyGems 4.0(TBD) The policy of RubyGems versioning
  • 24.
    • RubyGems alsohave HackerOne. • 3 people handle vulnerability issues and will release RubyGems by SemVer Policy like “2.7.7” from “2.7.6” • On the other hand, The Ruby core team will back port only vulnerability fixes by independent version like “2.6.5.1”, not “2.7.7” Security release of RubyGems
  • 25.
  • 26.
    • Removed deprecatedmethods. • Removed to support for < Ruby 2.2. • Added warnings of deprecated methods. • Removed deprecated options. • [CAUTION] `--ri` and `--rdoc` options What’s new in RubyGems3?
  • 27.
    • Surprisedly, RG2.7 still supports Ruby 1.8. Ruby 1.8 in 2018 ~/D/g/r/rubygems (2.7) > rg respond_to test/rubygems/test_gem_request_set_gem_dependency_api.rb 630: tf.close! if tf.respond_to? :close! test/rubygems/test_gem_source.rb 60: response.uri = URI('http://example') if response.respond_to? :uri test/rubygems/test_gem_package.rb 755: tf.close! if tf.respond_to? :close! test/rubygems/test_gem_util.rb 45: if File.respond_to?(:realpath) test/rubygems/test_gem_installer.rb 58: str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding 65:if Gem.respond_to?(:activate_bin_path) 893: skip unless "".respond_to?(:force_encoding) test/rubygems/test_gem_specification.rb 2305: s.required_rubygems_version = Gem::Requirement.new("> 0".freeze) if s.respond_to? :required_ruby 2316: if s.respond_to? :specification_version then …snip
  • 28.
    • We canuse Keywords argument, Refinement, Other cool features in RubyGems now. • Simple build matrix Only support Ruby 2.2+
  • 29.
    • It hasnon-compatible features. • Make enable as default for conservative option: https://github.com/rubygems/rubygems/pull/2233 • Make ruby gem install to user-install by default: https://github.com/rubygems/rubygems/issues/1394 • Executables in bin folder conflict with their gem versions: https://bugs.ruby-lang.org/issues/5060 • Behaviour changes with default gems installer: https:// github.com/rubygems/rubygems/pull/2166 What’s new in RubyGems4?
  • 30.
    • We gotthe installation time when already installed gems. • To use conservative is ignore re-install action. Make conservative option as default ~ > gem i rails clone http://rubyonrails.org -> /Users/hsbt/Documents/rubyonrails.org git ls-remote http://rubyonrails.org hg identify http://rubyonrails.org svn info http://rubyonrails.org error Could not find version control system: http://rubyonrails.org exists /Users/hsbt/Documents/github.com/rails/rails Successfully installed rails-5.2.0 1 gem installed ~ > gem i rails —conservative ~ >
  • 31.
    • Rubygems 4will install the all gems to `~/.gem` • Pros: Ruby in linux distribution has many of FAQ for gem installation for using `sudo`. This change resolve this issues. • Cons: Ruby version manager like rbenv is not support it. And This is big incompatible feature. Make `--user-install` as default
  • 32.
  • 33.
    • The vendoringtool of Ruby. • RubyGems couldn’t care dependency of Ruby libraries and isolate version managing with ruby process. • Bundler can do them with `Gemfile` What’s bundler? # frozen_string_literal: true source "https://rubygems.org" git_source(:github) { |repo| "https://github.com/#{repo}.git" } gemspec # We need a newish Rake since Active Job sets its test tasks' descriptions. gem "rake", ">= 11.1"
  • 35.
    • Maybe, It’sthis year. • We disabled the incompatible features like renaming `gems.rb` from `Gemfile` • We will release 1.17.0 at this week. After that, We bump its version to 2.0.0 and drop to support under the Ruby 2.2. When is coming Bundler 2?
  • 36.
  • 37.
    • We areworking to integrate RubyGems and Bundler. • RubyGems 3&4 drop to support under the Ruby 2.2. Because Bundler 1.x still supports Ruby 1.8 and 1.9. • I’m waiting to release Bundler 2 for this integration. After that, I will promote Bundler 2 as standard library of the Ruby Core. RubyGems/Bundler integration
  • 38.
    • Bundler waslocated rubygems repository as git submodule Bundler Integration(rubygems.rb) if USE_BUNDLER_FOR_GEMDEPS ENV["BUNDLE_GEMFILE"] ||= File.expand_path(path) require 'rubygems/user_interaction' Gem::DefaultUserInteraction.use_ui(ui) do require "bundler" @gemdeps = Bundler.setup Bundler.ui = nil @gemdeps.requested_specs.map(&:to_spec).sort_by(&:name) end else rs = Gem::RequestSet.new @gemdeps = rs.load_gemdeps path rs.resolve_current.map do |s| s.full_spec.tap(&:activate) end end
  • 39.
    • RubyGems 2.x,3.x uses Molinillo-0.5.7 • Bundler 1.16.x also uses Molinillo-0.6.4 • These are different versions and behavior of dependency resolver. Dependency Resolver incompatible ~/D/g/r/rubygems (master) > ls lib/rubygems/resolver/molinillo/lib/molinillo delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb dependency_graph errors.rb modules resolver.rb ~/D/g/b/bundler (master) > ls lib/bundler/vendor/molinillo/lib/molinillo compatibility.rb dependency_graph errors.rb modules resolver.rb delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb
  • 40.
  • 41.
    Executive Officer CPO(ChiefProductivity Officer) Director of Business Process Re-engineering Office Director of Technical Division at GMO Pepabo, Inc. @pepabo Hiroshi SHIBATA @hsbt https://www.hsbt.org
  • 42.
    Ruby is designedto make programmers happy. Yukihiro Matz Matsumoto