JRUBY, NOT JUST FOR HARD-
    HEADED PRAGMATISTS ANYMORE
                                                        IAN DEES • @UNDEES


                                                        WHISKEY-
                                                        CONF 2011
                                                                               ✹       ALSO
                                                                                       JR UBY




Hi, I’m Ian. Hope everyone’s having a great WhiskyConf. I hear we’ll have some JRuby material at
this conference as well. ;-)

http://www.flickr.com/photos/etnobofin/3786071796
Write a Compiler



Most JRuby talks are pragmatic: either a case study of how JRuby helped a project, or a specific
library that developers can use. This talk will be more theoretical. We’ll use JRuby as a vehicle to
explore an area of computer science; namely, writing a compiler.
The Three Paths
  Abstraction




                                                                             compilers




                                              Time
Depending on your path through computer science, you may have encountered the topic already.
Abstraction




                                                                                  compilers


                                                                    xor eax, eax




                       e-
                                                 Time
If, like me, you grew up on the hardware path, a compiler is the next logical step beyond learning
programming languages.
logic
                                              λx.x                 grammars
  Abstraction




                                                                                 compilers


                                                                   xor eax, eax




                       e-
                                                Time
If you took the software path and learned the theory of computation first, then compilers are a
practical application of those ideas.
logic
                                             λx.x                grammars
  Abstraction




                                                                               compilers


                                                                 xor eax, eax




                      e-
                                               Time
If you took the self-made path, you may or may not have encountered compilers by this point on
your journey. Either way, I hope the tools we talk about today pique your interest.
The Plan

           ✓Stay up late drinking whisky
           ✓Wear the T-shirt of the band that’s playing
           ✓Choose the wrong topic
           ✓Go at a pace that’s awkward for everyone
           ✓Code live on stage

The plan for this talk is to show up tired, wear a JRubyConf t-shirt (this is like showing up at a
concert wearing the t-shirt of the band that’s playing), choose a topic and pace different from all
the other talks, and introduce an element of uncertainty by coding on stage. (If you’re reading this,
I’ll show you GitHub commits instead.)
This might be a train wreck, but train wrecks can be entertaining as long as they’re toy trains.

http://www.flickr.com/photos/cianginty/3148870954
The Tools
           • Parslet
                http://kschiess.github.com/parslet

           • BiteScript
                https://github.com/headius/bitescript

           • Gritty
                https://github.com/undees/gritty

           • Graph
                http://rubygems.org/gems/graph

           • JRuby!
The first two tools are the ones we’ll use to write the compiler. The next two are the ones I used to
draw the pictures on the subsequent slides.
The Language



The language we’re going to be writing a compiler for is an extremely simple one.
Thnad
                                    (thank you, Dr. Seuss!)




Most of the real letters of the alphabet are already used for programming languages (C, D, J, K, R,
etc.). So we’ll use the fictional letter “Thnad,” which comes to us from Dr. Seuss.
function factorial(n) {
         if (eq(n, 1)) {
           1
         } else {
           times(n, factorial(minus(n, 1)))
         }
       }

       print(factorial(4))



Here’s the Thnad program we’d eventually like to compile. It has integers, function calls,
conditionals, and function definitions—and that’s about it.
1. Integers



One big step is to write a compiler top to bottom that parses an integer and outputs a working
Java .class file.
42




First, we’re going to write code that parses the text “42” into a Ruby hash representing the tree on
the right.
Following Along


            • Test → test_parser.rb
            • Code → parser.rb



Slides like this one will contain a link to the tests, code, or full examples for the steps we’ve just
been over.
Next, we’re going to transform that Ruby hash (which isn’t as useful by itself) into a custom Ruby
class that will eventually be able to output Java bytecode.
• Test → test_transform.rb
• Code → transform.rb
• Full Example
2. Function Calls



Now, let’s add the ability to compile function calls.
foo




We need to be able to parse identifiers; we’ll use these for function and parameter names.
As we did with integers, we want to transform the first hash representation into a more specific
Ruby object.
(42, foo)




Next, we’ll parse an argument list, which is a set of parentheses containing integers and parameter
names.
We’ll need code to transform that into an array of the Ruby objects we’ve already defined.
baz(42, foo)




Now, we’re finally ready to parse a function call...
...and transform it into a Ruby object that will be able to emit bytecode.
• Full Example
3. Conditionals



The next big piece of our compiler is the conditional expression.
if (0) {
                42
              } else {
                667
              }




Here’s the kind of conditional statement we’d like to be able to parse. The “else” clause is
mandatory (I hope Zed Shaw will approve).
Paradoxically, the more complicated the expression we parse, the more our tree of custom Ruby
classes resembles the Ruby hash we started with.
• Full Example
4. Function Definitions



The last piece we need for our factorial Thnad program is the ability to compile function definitions.
function foo(x) {
       5
     }




A function definition is the keyword “function” followed by a list of parameters in parentheses, then
a computation between braces.
And here’s the resulting tree of Ruby objects.
• Full Example
Marc-André Cournoyer

Marc-André Cournoyer’s book on creating your own programming language was helpful during this
process as a way of “checking my answers” (just like checking your answers to today’s crossword
puzzle by looking in tomorrow’s paper).
Code
                           https://github.com/undees/thnad




The full compiler, complete with bytecode generation, is available here.
For more about JRuby (which made this whole exercise possible in half an hour), see our book at
http://pragprog.com/titles/jruby.

JRuby, Not Just For Hard-Headed Pragmatists Anymore

  • 1.
    JRUBY, NOT JUSTFOR HARD- HEADED PRAGMATISTS ANYMORE IAN DEES • @UNDEES WHISKEY- CONF 2011 ✹ ALSO JR UBY Hi, I’m Ian. Hope everyone’s having a great WhiskyConf. I hear we’ll have some JRuby material at this conference as well. ;-) http://www.flickr.com/photos/etnobofin/3786071796
  • 2.
    Write a Compiler MostJRuby talks are pragmatic: either a case study of how JRuby helped a project, or a specific library that developers can use. This talk will be more theoretical. We’ll use JRuby as a vehicle to explore an area of computer science; namely, writing a compiler.
  • 3.
    The Three Paths Abstraction compilers Time Depending on your path through computer science, you may have encountered the topic already.
  • 4.
    Abstraction compilers xor eax, eax e- Time If, like me, you grew up on the hardware path, a compiler is the next logical step beyond learning programming languages.
  • 5.
    logic λx.x grammars Abstraction compilers xor eax, eax e- Time If you took the software path and learned the theory of computation first, then compilers are a practical application of those ideas.
  • 6.
    logic λx.x grammars Abstraction compilers xor eax, eax e- Time If you took the self-made path, you may or may not have encountered compilers by this point on your journey. Either way, I hope the tools we talk about today pique your interest.
  • 7.
    The Plan ✓Stay up late drinking whisky ✓Wear the T-shirt of the band that’s playing ✓Choose the wrong topic ✓Go at a pace that’s awkward for everyone ✓Code live on stage The plan for this talk is to show up tired, wear a JRubyConf t-shirt (this is like showing up at a concert wearing the t-shirt of the band that’s playing), choose a topic and pace different from all the other talks, and introduce an element of uncertainty by coding on stage. (If you’re reading this, I’ll show you GitHub commits instead.)
  • 8.
    This might bea train wreck, but train wrecks can be entertaining as long as they’re toy trains. http://www.flickr.com/photos/cianginty/3148870954
  • 9.
    The Tools • Parslet http://kschiess.github.com/parslet • BiteScript https://github.com/headius/bitescript • Gritty https://github.com/undees/gritty • Graph http://rubygems.org/gems/graph • JRuby! The first two tools are the ones we’ll use to write the compiler. The next two are the ones I used to draw the pictures on the subsequent slides.
  • 10.
    The Language The languagewe’re going to be writing a compiler for is an extremely simple one.
  • 11.
    Thnad (thank you, Dr. Seuss!) Most of the real letters of the alphabet are already used for programming languages (C, D, J, K, R, etc.). So we’ll use the fictional letter “Thnad,” which comes to us from Dr. Seuss.
  • 12.
    function factorial(n) { if (eq(n, 1)) { 1 } else { times(n, factorial(minus(n, 1))) } } print(factorial(4)) Here’s the Thnad program we’d eventually like to compile. It has integers, function calls, conditionals, and function definitions—and that’s about it.
  • 13.
    1. Integers One bigstep is to write a compiler top to bottom that parses an integer and outputs a working Java .class file.
  • 14.
    42 First, we’re goingto write code that parses the text “42” into a Ruby hash representing the tree on the right.
  • 15.
    Following Along • Test → test_parser.rb • Code → parser.rb Slides like this one will contain a link to the tests, code, or full examples for the steps we’ve just been over.
  • 16.
    Next, we’re goingto transform that Ruby hash (which isn’t as useful by itself) into a custom Ruby class that will eventually be able to output Java bytecode.
  • 17.
    • Test →test_transform.rb • Code → transform.rb • Full Example
  • 18.
    2. Function Calls Now,let’s add the ability to compile function calls.
  • 19.
    foo We need tobe able to parse identifiers; we’ll use these for function and parameter names.
  • 20.
    As we didwith integers, we want to transform the first hash representation into a more specific Ruby object.
  • 21.
    (42, foo) Next, we’llparse an argument list, which is a set of parentheses containing integers and parameter names.
  • 22.
    We’ll need codeto transform that into an array of the Ruby objects we’ve already defined.
  • 23.
    baz(42, foo) Now, we’refinally ready to parse a function call...
  • 24.
    ...and transform itinto a Ruby object that will be able to emit bytecode.
  • 25.
  • 26.
    3. Conditionals The nextbig piece of our compiler is the conditional expression.
  • 27.
    if (0) { 42 } else { 667 } Here’s the kind of conditional statement we’d like to be able to parse. The “else” clause is mandatory (I hope Zed Shaw will approve).
  • 28.
    Paradoxically, the morecomplicated the expression we parse, the more our tree of custom Ruby classes resembles the Ruby hash we started with.
  • 29.
  • 30.
    4. Function Definitions Thelast piece we need for our factorial Thnad program is the ability to compile function definitions.
  • 31.
    function foo(x) { 5 } A function definition is the keyword “function” followed by a list of parameters in parentheses, then a computation between braces.
  • 32.
    And here’s theresulting tree of Ruby objects.
  • 33.
  • 34.
    Marc-André Cournoyer Marc-André Cournoyer’sbook on creating your own programming language was helpful during this process as a way of “checking my answers” (just like checking your answers to today’s crossword puzzle by looking in tomorrow’s paper).
  • 35.
    Code https://github.com/undees/thnad The full compiler, complete with bytecode generation, is available here.
  • 36.
    For more aboutJRuby (which made this whole exercise possible in half an hour), see our book at http://pragprog.com/titles/jruby.