🌱 idlecycles

Chapter 4: Mini Notation Parsing

One very distinct feature of tidal is its mini notation. Let's build a simplistic version from scratch! I'll use a similar tactic than with my lisp parser, which is heavily inspired by A Compiler From Scratch

Our goal

We want to be able to write something like:

which should be equivalent to:

cat("cyan", "magenta", seq("yellow", "white")).fast(5);

To be able to do this, we need to parse the mini notation string and somehow construct the appropriate function calls.

The Tokenizer

We start with a list of valid tokens, which are the smallest meaningful chunks our code can have:

Here, we map from a token identifier to the regex that parses it.

With it, we can tokenize a string like this:

Now we can use that function repeatedly to tokenize a whole string:

output:

[
  { "type": "open_cat", "value": "<" },
  { "type": "plain", "value": "cyan" },
  { "type": "plain", "value": "magenta" },
  { "type": "close_cat", "value": ">" }
]
    

Here, we are taking token by token out of the string, until it is empty. That's already a fine tokenizer, let's move on to the parser!

The Parser

After the code has been tokenized, we can construct a tree. Mini notation supports arbitrary levels of nesting, for example:

Notated as a JSON object, we'd expect the structure to look like:

  {
    "type": "cat",
    "args": [
      { "type": "plain", "value": "cyan" },
      {
        "type": "seq",
        "args": [
          { "type": "plain", "value": "magenta" },
          {
            "type": "seq",
            "args": [
              { "type": "plain", "value": "white" },
              { "type": "plain", "value": "black" }
            ]
          },
          { "type": "plain", "value": "yellow" }
        ]
      }
    ]
  }
  

Here's a minimal implementation:

you can inspect the logged ast (abstract syntax tree) in the browser console!

From a Tree to a Pattern

Now having a tree that represents the layers of nesting in our syntax, the last step is to convert it into pattern function calls:

This is all we need! Now we can tie the whole thing together:

And that's a basic implementation of the mini notation! It certainly doesn't cover all features, but it should give you the basic idea.

chapter 5: Operating on Values

all chapters


show page source

    

license: AGPL-3.0

back to garten.salat