.gitattributes

@@ -0,0 +1 @@
+bin/* linguist-language=Ruby

.rubocop.yml

@@ -7,7 +7,7 @@ AllCops:
   SuggestExtensions: false
   TargetRubyVersion: 2.7
   Exclude:
-  - '{bin,coverage,pkg,test/fixtures,vendor,tmp}/**/*'
+  - '{.git,.github,bin,coverage,pkg,test/fixtures,vendor,tmp}/**/*'
   - test.rb
 
 Layout/LineLength:

CHANGELOG.md

@@ -6,6 +6,30 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [5.0.0] - 2022-11-09
+
+### Added
+
+- Every node now implements the `#copy(**)` method, which provides a copy of the node with the given attributes replaced.
+- Every node now implements the `#===(other)` method, which checks if the given node matches the current node for all attributes except for comments and location.
+- There is a new `SyntaxTree::Visitor::MutationVisitor` and its convenience method `SyntaxTree.mutation` which can be used to mutate a syntax tree. For details on how to use this visitor, check the README.
+
+### Changed
+
+- Nodes no longer have a `comments:` keyword on their initializers. By default, they initialize to an empty array. If you were previously passing comments into the initializer, you should now create the node first, then call `node.comments.concat` to add your comments.
+- A lot of nodes have been folded into other nodes to make it easier to interact with the AST. This means that a lot of visit methods have been removed from the visitor and a lot of class definitions are no longer present. This also means that the nodes that received more function now have additional methods or fields to be able to differentiate them. Note that none of these changes have resulted in different formatting. The changes are listed below:
+  - `IfMod`, `UnlessMod`, `WhileMod`, `UntilMod` have been folded into `IfNode`, `UnlessNode`, `WhileNode`, and `UntilNode`. Each of the nodes now have a `modifier?` method to tell if it was originally in the modifier form. Consequently, the `visit_if_mod`, `visit_unless_mod`, `visit_while_mod`, and `visit_until_mod` methods have been removed from the visitor.
+  - `VarAlias` is no longer a node, and the `Alias` node has been renamed. They have been folded into the `AliasNode` node. The `AliasNode` node now has a `var_alias?` method to tell you if it is aliasing a global variable. Consequently, the `visit_var_alias` method has been removed from the visitor interface. If you were previously using this method, you should now use `visit_alias` instead.
+  - `Yield0` is no longer a node, and the `Yield` node has been renamed. They has been folded into the `YieldNode` node. The `YieldNode` node can now have its `arguments` field be `nil`. Consequently, the `visit_yield0` method has been removed from the visitor interface. If you were previously using this method, you should now use `visit_yield` instead.
+  - `FCall` is no longer a node, and the `Call` node has been renamed. They have been folded into the `CallNode` node. The `CallNode` node can now have its `receiver` and `operator` fields be `nil`. Consequently, the `visit_fcall` method has been removed from the visitor interface. If you were previously using this method, you should now use `visit_call` instead.
+  - `Dot2` and `Dot3` are no longer nodes. Instead they have become a single new `RangeNode` node. This node looks the same as `Dot2` and `Dot3`, except that it additionally has an `operator` field that contains the operator that created the node. Consequently, the `visit_dot2` and `visit_dot3` methods have been removed from the visitor interface. If you were previously using these methods, you should now use `visit_range` instead.
+  - `Def`, `DefEndless`, and `Defs` have been folded into the `DefNode` node. The `DefNode` node now has the `target` and `operator` fields which originally came from `Defs` which can both be `nil`. It also now has an `endless?` method on it to tell if the original node was found in the endless form. Finally the `bodystmt` field can now either be a `BodyStmt` as it was or any other kind of node since that was the body of the `DefEndless` node. The `visit_defs` and `visit_def_endless` methods on the visitor have therefore been removed.
+  - `DoBlock` and `BraceBlock` have now been folded into a `BlockNode` node. The `BlockNode` node now has a `keywords?` method on it that returns true if the block was constructed with the `do`..`end` keywords. The `visit_do_block` and `visit_brace_block` methods on the visitor have therefore been removed and replaced with the `visit_block` method.
+  - `Return0` is no longer a node, and the `Return` node has been renamed. They have been folded into the `ReturnNode` node. The `ReturnNode` node can now have its `arguments` field be `nil`. Consequently, the `visit_return0` method has been removed from the visitor interface. If you were previously using this method, you should now use `visit_return` instead.
+- The `ArgsForward`, `Redo`, `Retry`, and `ZSuper` nodes no longer have `value` fields associated with them (which were always string literals corresponding to the keyword being used).
+- The `Command` and `CommandCall` nodes now has `block` attributes on them. These attributes are used in the place where you would previously have had a `MethodAddBlock` structure. Where before the `MethodAddBlock` would have the command and block as its two children, you now just have one command node with the `block` attribute set to the `Block` node.
+- Previously the formatting options were defined on an unfrozen hash called `SyntaxTree::Formatter::OPTIONS`. It was globally mutable, which made it impossible to reference from within a Ractor. As such, it has now been replaced with `SyntaxTree::Formatter::Options.new` which creates a new options object instance that can be modified without impacting global state. As a part of this change, formatting can now be performed from within a non-main Ractor. In order to check if the `plugin/single_quotes` plugin has been loaded, check if `SyntaxTree::Formatter::SINGLE_QUOTES` is defined. In order to check if the `plugin/trailing_comma` plugin has been loaded, check if `SyntaxTree::Formatter::TRAILING_COMMA` is defined.
+
 ## [4.3.0] - 2022-10-28
 
 ### Added
@@ -426,7 +450,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.3.0...HEAD
+[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v5.0.0...HEAD
+[5.0.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.3.0...v5.0.0
 [4.3.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.2.0...v4.3.0
 [4.2.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.1.0...v4.2.0
 [4.1.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.0.2...v4.1.0

Gemfile.lock

@@ -1,8 +1,8 @@
 PATH
   remote: .
   specs:
-    syntax_tree (4.3.0)
-      prettier_print (>= 1.0.2)
+    syntax_tree (5.0.0)
+      prettier_print (>= 1.1.0)
 
 GEM
   remote: https://rubygems.org/
@@ -14,12 +14,12 @@ GEM
     parallel (1.22.1)
     parser (3.1.2.1)
       ast (~> 2.4.1)
-    prettier_print (1.0.2)
+    prettier_print (1.1.0)
     rainbow (3.1.1)
     rake (13.0.6)
     regexp_parser (2.6.0)
     rexml (3.2.5)
-    rubocop (1.37.1)
+    rubocop (1.38.0)
       json (~> 2.3)
       parallel (~> 1.10)
       parser (>= 3.1.2.1)

README.md

@@ -27,23 +27,29 @@ It is built with only standard library dependencies. It additionally ships with
   - [SyntaxTree.read(filepath)](#syntaxtreereadfilepath)
   - [SyntaxTree.parse(source)](#syntaxtreeparsesource)
   - [SyntaxTree.format(source)](#syntaxtreeformatsource)
+  - [SyntaxTree.mutation(&block)](#syntaxtreemutationblock)
   - [SyntaxTree.search(source, query, &block)](#syntaxtreesearchsource-query-block)
 - [Nodes](#nodes)
   - [child_nodes](#child_nodes)
+  - [copy(**attrs)](#copyattrs)
   - [Pattern matching](#pattern-matching)
   - [pretty_print(q)](#pretty_printq)
   - [to_json(*opts)](#to_jsonopts)
   - [format(q)](#formatq)
+  - [===(other)](#other)
   - [construct_keys](#construct_keys)
 - [Visitor](#visitor)
   - [visit_method](#visit_method)
   - [BasicVisitor](#basicvisitor)
+  - [MutationVisitor](#mutationvisitor)
+  - [WithEnvironment](#withenvironment)
 - [Language server](#language-server)
   - [textDocument/formatting](#textdocumentformatting)
   - [textDocument/inlayHint](#textdocumentinlayhint)
   - [syntaxTree/visualizing](#syntaxtreevisualizing)
-- [Plugins](#plugins)
-  - [Customization](#customization)
+- [Customization](#customization)
+  - [Ignoring code](#ignoring-code)
+  - [Plugins](#plugins)
   - [Languages](#languages)
 - [Integration](#integration)
   - [Rake](#rake)
@@ -332,6 +338,10 @@ This function takes an input string containing Ruby code and returns the syntax
 
 This function takes an input string containing Ruby code, parses it into its underlying syntax tree, and formats it back out to a string. You can optionally pass a second argument to this method as well that is the maximum width to print. It defaults to `80`.
 
+### SyntaxTree.mutation(&block)
+
+This function yields a new mutation visitor to the block, and then returns the initialized visitor. It's effectively a shortcut for creating a `SyntaxTree::Visitor::MutationVisitor` without having to remember the class name. For more information on that visitor, see the definition below.
+
 ### SyntaxTree.search(source, query, &block)
 
 This function takes an input string containing Ruby code, an input string containing a valid Ruby `in` clause expression that can be used to match against nodes in the tree (can be generated using `stree expr`, `stree match`, or `Node#construct_keys`), and a block. Each node that matches the given query will be yielded to the block. The block will receive the node as its only argument.
@@ -350,6 +360,20 @@ program.child_nodes.first.child_nodes.first
 # => (binary (int "1") :+ (int "1"))
 ```
 
+### copy(**attrs)
+
+This method returns a copy of the node, with the given attributes replaced.
+
+```ruby
+program = SyntaxTree.parse("1 + 1")
+
+binary = program.statements.body.first
+# => (binary (int "1") + (int "1"))
+
+binary.copy(operator: :-)
+# => (binary (int "1") - (int "1"))
+```
+
 ### Pattern matching
 
 Pattern matching is another way to descend the tree which is more specific than using `child_nodes`. Using Ruby's built-in pattern matching, you can extract the same information but be as specific about your constraints as you like. For example, with minimal constraints:
@@ -407,6 +431,18 @@ formatter.output.join
 # => "1 + 1"
 ```
 
+### ===(other)
+
+Every node responds to `===`, which is used to check if the given other node matches all of the attributes of the current node except for location and comments. For example:
+
+```ruby
+program1 = SyntaxTree.parse("1 + 1")
+program2 = SyntaxTree.parse("1 + 1")
+
+program1 === program2
+# => true
+```
+
 ### construct_keys
 
 Every node responds to `construct_keys`, which will return a string that contains a Ruby pattern-matching expression that could be used to match against the current node. It's meant to be used in tooling and through the CLI mostly.
@@ -495,6 +531,42 @@ end
 
 The visitor defined above will error out unless it's only visiting a `SyntaxTree::Int` node. This is useful in a couple of ways, e.g., if you're trying to define a visitor to handle the whole tree but it's currently a work-in-progress.
 
+### MutationVisitor
+
+The `MutationVisitor` is a visitor that can be used to mutate the tree. It works by defining a default `visit_*` method that returns a copy of the given node with all of its attributes visited. This new node will replace the old node in the tree. Typically, you use the `#mutate` method on it to define mutations using patterns. For example:
+
+```ruby
+# Create a new visitor
+visitor = SyntaxTree::Visitor::MutationVisitor.new
+
+# Specify that it should mutate If nodes with assignments in their predicates
+visitor.mutate("IfNode[predicate: Assign | OpAssign]") do |node|
+  # Get the existing If's predicate node
+  predicate = node.predicate
+
+  # Create a new predicate node that wraps the existing predicate node
+  # in parentheses
+  predicate =
+    SyntaxTree::Paren.new(
+      lparen: SyntaxTree::LParen.default,
+      contents: predicate,
+      location: predicate.location
+    )
+
+  # Return a copy of this node with the new predicate
+  node.copy(predicate: predicate)
+end
+
+source = "if a = 1; end"
+program = SyntaxTree.parse(source)
+
+SyntaxTree::Formatter.format(source, program)
+# => "if a = 1\nend\n"
+
+SyntaxTree::Formatter.format(source, program.accept(visitor))
+# => "if (a = 1)\nend\n"
+```
+
 ### WithEnvironment
 
 The `WithEnvironment` module can be included in visitors to automatically keep track of local variables and arguments
@@ -506,13 +578,13 @@ class MyVisitor < Visitor
   include WithEnvironment
 
   def visit_ident(node)
-    # find_local will return a Local for any local variables or arguments present in the current environment or nil if
-    # the identifier is not a local
+    # find_local will return a Local for any local variables or arguments
+    # present in the current environment or nil if the identifier is not a local
     local = current_environment.find_local(node)
 
-    puts local.type # print the type of the local (:variable or :argument)
-    puts local.definitions # print the array of locations where this local is defined
-    puts local.usages # print the array of locations where this local occurs
+    puts local.type # the type of the local (:variable or :argument)
+    puts local.definitions # the array of locations where this local is defined
+    puts local.usages # the array of locations where this local occurs
   end
 end
 ```
@@ -549,18 +621,45 @@ Implicity, the `2 * 3` is going to be executed first because the `*` operator ha
 
 The language server additionally includes this custom request to return a textual representation of the syntax tree underlying the source code of a file. Language server clients can use this to (for example) open an additional tab with this information displayed.
 
-## Plugins
+## Customization
+
+There are multiple ways to customize Syntax Tree's behavior when parsing and formatting code. You can ignore certain sections of the source code, you can register plugins to provide custom formatting behavior, and you can register additional languages to be parsed and formatted.
+
+### Ignoring code
+
+To ignore a section of source code, you can use a special `# stree-ignore` comment. This comment should be placed immediately above the code that you want to ignore. For example:
+
+```ruby
+numbers = [
+  10000,
+  20000,
+  30000
+]
+```
+
+Normally the snippet above would be formatted as `numbers = [10_000, 20_000, 30_000]`. However, sometimes you want to keep the original formatting to improve readability or maintainability. In that case, you can put the ignore comment before it, as in:
+
+```ruby
+# stree-ignore
+numbers = [
+  10000,
+  20000,
+  30000
+]
+```
+
+Now when Syntax Tree goes to format that code, it will copy the source code exactly as it is, including the newlines and indentation.
 
-You can register additional customization and additional languages that can flow through the same CLI with Syntax Tree's plugin system. When invoking the CLI, you pass through the list of plugins with the `--plugins` options to the commands that accept them. They should be a comma-delimited list. When the CLI first starts, it will require the files corresponding to those names.
+### Plugins
 
-### Customization
+You can register additional customization that can flow through the same CLI with Syntax Tree's plugin system. When invoking the CLI, you pass through the list of plugins with the `--plugins` options to the commands that accept them. They should be a comma-delimited list. When the CLI first starts, it will require the files corresponding to those names.
 
-To register additional customization, define a file somewhere in your load path named `syntax_tree/my_plugin`. Then when invoking the CLI, you will pass `--plugins=my_plugin`. To require multiple, separate them by a comma. In this way, you can modify Syntax Tree however you would like. Some plugins ship with Syntax Tree itself. They are:
+To register plugins, define a file somewhere in your load path named `syntax_tree/my_plugin`. Then when invoking the CLI, you will pass `--plugins=my_plugin`. To require multiple, separate them by a comma. In this way, you can modify Syntax Tree however you would like. Some plugins ship with Syntax Tree itself. They are:
 
 * `plugin/single_quotes` - This will change all of your string literals to use single quotes instead of the default double quotes.
 * `plugin/trailing_comma` - This will put trailing commas into multiline array literals, hash literals, and method calls that can support trailing commas.
 
-If you're using Syntax Tree as a library, you should require those files directly.
+If you're using Syntax Tree as a library, you can require those files directly or manually pass those options to the formatter initializer through the `SyntaxTree::Formatter::Options` class.
 
 ### Languages
 

lib/syntax_tree.rb

@@ -16,6 +16,7 @@
 require_relative "syntax_tree/visitor/field_visitor"
 require_relative "syntax_tree/visitor/json_visitor"
 require_relative "syntax_tree/visitor/match_visitor"
+require_relative "syntax_tree/visitor/mutation_visitor"
 require_relative "syntax_tree/visitor/pretty_print_visitor"
 require_relative "syntax_tree/visitor/environment"
 require_relative "syntax_tree/visitor/with_environment"
@@ -53,14 +54,25 @@ def self.parse(source)
   end
 
   # Parses the given source and returns the formatted source.
-  def self.format(source, maxwidth = DEFAULT_PRINT_WIDTH)
-    formatter = Formatter.new(source, [], maxwidth)
+  def self.format(
+    source,
+    maxwidth = DEFAULT_PRINT_WIDTH,
+    options: Formatter::Options.new
+  )
+    formatter = Formatter.new(source, [], maxwidth, options: options)
     parse(source).format(formatter)
 
     formatter.flush
     formatter.output.join
   end
 
+  # A convenience method for creating a new mutation visitor.
+  def self.mutation
+    visitor = Visitor::MutationVisitor.new
+    yield visitor
+    visitor
+  end
+
   # Returns the source from the given filepath taking into account any potential
   # magic encoding comments.
   def self.read(filepath)