Final tests, documentation, changelog for mutations

Author: kddnewton

CHANGELOG.md

@@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+### Added
+
+- Every node now implements the `#copy(**)` method, which provides a copy of the node with the given attributes replaced.
+- Every node now implements the `#===(other)` method, which checks if the given node matches the current node for all attributes except for comments and location.
+- There is a new `SyntaxTree::Visitor::MutationVisitor` and its convenience method `SyntaxTree.mutation` which can be used to mutate a syntax tree. For details on how to use this visitor, check the README.
+
 ### Changed
 
 - Nodes no longer have a `comments:` keyword on their initializers. By default, they initialize to an empty array. If you were previously passing comments into the initializer, you should now create the node first, then call `node.comments.concat` to add your comments.

README.md

@@ -27,17 +27,21 @@ It is built with only standard library dependencies. It additionally ships with
   - [SyntaxTree.read(filepath)](#syntaxtreereadfilepath)
   - [SyntaxTree.parse(source)](#syntaxtreeparsesource)
   - [SyntaxTree.format(source)](#syntaxtreeformatsource)
+  - [SyntaxTree.mutation(&block)](#syntaxtreemutationblock)
   - [SyntaxTree.search(source, query, &block)](#syntaxtreesearchsource-query-block)
 - [Nodes](#nodes)
   - [child_nodes](#child_nodes)
+  - [copy(**)](#copy)
   - [Pattern matching](#pattern-matching)
   - [pretty_print(q)](#pretty_printq)
   - [to_json(*opts)](#to_jsonopts)
   - [format(q)](#formatq)
+  - [===(other)](#other)
   - [construct_keys](#construct_keys)
 - [Visitor](#visitor)
   - [visit_method](#visit_method)
   - [BasicVisitor](#basicvisitor)
+  - [MutationVisitor](#mutationvisitor)
 - [Language server](#language-server)
   - [textDocument/formatting](#textdocumentformatting)
   - [textDocument/inlayHint](#textdocumentinlayhint)
@@ -332,6 +336,10 @@ This function takes an input string containing Ruby code and returns the syntax
 
 This function takes an input string containing Ruby code, parses it into its underlying syntax tree, and formats it back out to a string. You can optionally pass a second argument to this method as well that is the maximum width to print. It defaults to `80`.
 
+### SyntaxTree.mutation(&block)
+
+This function yields a new mutation visitor to the block, and then returns the initialized visitor. It's effectively a shortcut for creating a `SyntaxTree::Visitor::MutationVisitor` without having to remember the class name. For more information on that visitor, see the definition below.
+
 ### SyntaxTree.search(source, query, &block)
 
 This function takes an input string containing Ruby code, an input string containing a valid Ruby `in` clause expression that can be used to match against nodes in the tree (can be generated using `stree expr`, `stree match`, or `Node#construct_keys`), and a block. Each node that matches the given query will be yielded to the block. The block will receive the node as its only argument.
@@ -350,6 +358,20 @@ program.child_nodes.first.child_nodes.first
 # => (binary (int "1") :+ (int "1"))
 ```
 
+### copy
+
+This method returns a copy of the node, with the given attributes replaced.
+
+```ruby
+program = SyntaxTree.parse("1 + 1")
+
+binary = program.statements.body.first
+# => (binary (int "1") + (int "1"))
+
+binary.copy(operator: :-)
+# => (binary (int "1") - (int "1"))
+```
+
 ### Pattern matching
 
 Pattern matching is another way to descend the tree which is more specific than using `child_nodes`. Using Ruby's built-in pattern matching, you can extract the same information but be as specific about your constraints as you like. For example, with minimal constraints:
@@ -407,6 +429,18 @@ formatter.output.join
 # => "1 + 1"
 ```
 
+### ===(other)
+
+Every node responds to `===`, which is used to check if the given other node matches all of the attributes of the current node except for location and comments. For example:
+
+```ruby
+program1 = SyntaxTree.parse("1 + 1")
+program2 = SyntaxTree.parse("1 + 1")
+
+program1 === program2
+# => true
+```
+
 ### construct_keys
 
 Every node responds to `construct_keys`, which will return a string that contains a Ruby pattern-matching expression that could be used to match against the current node. It's meant to be used in tooling and through the CLI mostly.
@@ -495,6 +529,42 @@ end
 
 The visitor defined above will error out unless it's only visiting a `SyntaxTree::Int` node. This is useful in a couple of ways, e.g., if you're trying to define a visitor to handle the whole tree but it's currently a work-in-progress.
 
+### MutationVisitor
+
+The `MutationVisitor` is a visitor that can be used to mutate the tree. It works by defining a default `visit_*` method that returns a copy of the given node with all of its attributes visited. This new node will replace the old node in the tree. Typically, you use the `#mutate` method on it to define mutations using patterns. For example:
+
+```ruby
+# Create a new visitor
+visitor = SyntaxTree::Visitor::MutationVisitor.new
+
+# Specify that it should mutate If nodes with assignments in their predicates
+visitor.mutate("If[predicate: Assign | OpAssign]") do |node|
+  # Get the existing If's predicate node
+  predicate = node.predicate
+
+  # Create a new predicate node that wraps the existing predicate node
+  # in parentheses
+  predicate =
+    SyntaxTree::Paren.new(
+      lparen: SyntaxTree::LParen.default,
+      contents: predicate,
+      location: predicate.location
+    )
+
+  # Return a copy of this node with the new predicate
+  node.copy(predicate: predicate)
+end
+
+source = "if a = 1; end"
+program = SyntaxTree.parse(source)
+
+SyntaxTree::Formatter.format(source, program)
+# => "if a = 1\nend\n"
+
+SyntaxTree::Formatter.format(source, program.accept(visitor))
+# => "if (a = 1)\nend\n"
+```
+
 ### WithEnvironment
 
 The `WithEnvironment` module can be included in visitors to automatically keep track of local variables and arguments

lib/syntax_tree.rb

@@ -62,6 +62,13 @@ def self.format(source, maxwidth = DEFAULT_PRINT_WIDTH)
     formatter.output.join
   end
 
+  # A convenience method for creating a new mutation visitor.
+  def self.mutation
+    visitor = Visitor::MutationVisitor.new
+    yield visitor
+    visitor
+  end
+
   # Returns the source from the given filepath taking into account any potential
   # magic encoding comments.
   def self.read(filepath)

lib/syntax_tree/node.rb

@@ -84,6 +84,9 @@ def self.fixed(line:, char:, column:)
       )
     end
 
+    # A convenience method that is typically used when you don't care about the
+    # location of a node, but need to create a Location instance to pass to a
+    # constructor.
     def self.default
       new(
         start_line: 1,
@@ -7239,6 +7242,15 @@ def format(q)
     def ===(other)
       other.is_a?(LBrace) && value === other.value
     end
+
+    # Because some nodes keep around a { token so that comments can be attached
+    # to it if they occur in the source, oftentimes an LBrace is a child of
+    # another node. This means it's required at initialization time. To make it
+    # easier to create LBrace nodes without any specific value, this method
+    # provides a default node.
+    def self.default
+      new(value: "{", location: Location.default)
+    end
   end
 
   # LBracket represents the use of a left bracket, i.e., [.
@@ -7287,6 +7299,15 @@ def format(q)
     def ===(other)
       other.is_a?(LBracket) && value === other.value
     end
+
+    # Because some nodes keep around a [ token so that comments can be attached
+    # to it if they occur in the source, oftentimes an LBracket is a child of
+    # another node. This means it's required at initialization time. To make it
+    # easier to create LBracket nodes without any specific value, this method
+    # provides a default node.
+    def self.default
+      new(value: "[", location: Location.default)
+    end
   end
 
   # LParen represents the use of a left parenthesis, i.e., (.
@@ -7335,6 +7356,15 @@ def format(q)
     def ===(other)
       other.is_a?(LParen) && value === other.value
     end
+
+    # Because some nodes keep around a ( token so that comments can be attached
+    # to it if they occur in the source, oftentimes an LParen is a child of
+    # another node. This means it's required at initialization time. To make it
+    # easier to create LParen nodes without any specific value, this method
+    # provides a default node.
+    def self.default
+      new(value: "(", location: Location.default)
+    end
   end
 
   # MAssign is a parent node of any kind of multiple assignment. This includes

lib/syntax_tree/visitor/mutation_visitor.rb

@@ -5,30 +5,33 @@ class Visitor
     # This visitor walks through the tree and copies each node as it is being
     # visited. This is useful for mutating the tree before it is formatted.
     class MutationVisitor < BasicVisitor
-      # Here we maintain a stack of parent nodes so that it's easy to reflect on
-      # the context of a given node while mutating it.
-      attr_reader :stack
+      attr_reader :mutations
 
       def initialize
-        @stack = []
+        @mutations = []
       end
 
-      # This is the main entrypoint that's going to be called when we're
-      # recursing down through the tree.
+      # Create a new mutation based on the given query that will mutate the node
+      # using the given block. The block should return a new node that will take
+      # the place of the given node in the tree. These blocks frequently make
+      # use of the `copy` method on nodes to create a new node with the same
+      # properties as the original node.
+      def mutate(query, &block)
+        mutations << [Pattern.new(query).compile, block]
+      end
+
+      # This is the base visit method for each node in the tree. It first
+      # creates a copy of the node using the visit_* methods defined below. Then
+      # it checks each mutation in sequence and calls it if it finds a match.
       def visit(node)
         return unless node
-
-        stack << node
         result = node.accept(self)
 
-        stack.pop
-        result
-      end
+        mutations.each do |(pattern, mutation)|
+          result = mutation.call(result) if pattern.call(result)
+        end
 
-      # This is a small helper to visit an array of nodes and return the result
-      # of visiting them all.
-      def visit_all(nodes)
-        nodes.map { |node| visit(node) }
+        result
       end
 
       # Visit a BEGINBlock node.
@@ -435,6 +438,7 @@ def visit_ident(node)
       # Visit a If node.
       def visit_if(node)
         node.copy(
+          predicate: visit(node.predicate),
           statements: visit(node.statements),
           consequent: visit(node.consequent)
         )
@@ -822,14 +826,18 @@ def visit_undef(node)
       # Visit a Unless node.
       def visit_unless(node)
         node.copy(
+          predicate: visit(node.predicate),
           statements: visit(node.statements),
           consequent: visit(node.consequent)
         )
       end
 
       # Visit a Until node.
       def visit_until(node)
-        node.copy(statements: visit(node.statements))
+        node.copy(
+          predicate: visit(node.predicate),
+          statements: visit(node.statements)
+        )
       end
 
       # Visit a VarField node.
@@ -868,7 +876,10 @@ def visit_when(node)
 
       # Visit a While node.
       def visit_while(node)
-        node.copy(statements: visit(node.statements))
+        node.copy(
+          predicate: visit(node.predicate),
+          statements: visit(node.statements)
+        )
       end
 
       # Visit a Word node.

test/mutation_test.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+require_relative "test_helper"
+
+module SyntaxTree
+  class MutationTest < Minitest::Test
+    def test_mutates_based_on_patterns
+      source = <<~RUBY
+        if a = b
+          c
+        end
+      RUBY
+
+      expected = <<~RUBY
+        if (a = b)
+          c
+        end
+      RUBY
+
+      program = SyntaxTree.parse(source).accept(build_mutation)
+      assert_equal(expected, SyntaxTree::Formatter.format(source, program))
+    end
+
+    private
+
+    def build_mutation
+      SyntaxTree.mutation do |mutation|
+        mutation.mutate("If[predicate: Assign | OpAssign]") do |node|
+          # Get the existing If's predicate node
+          predicate = node.predicate
+
+          # Create a new predicate node that wraps the existing predicate node
+          # in parentheses
+          predicate =
+            SyntaxTree::Paren.new(
+              lparen: SyntaxTree::LParen.default,
+              contents: predicate,
+              location: predicate.location
+            )
+
+          # Return a copy of this node with the new predicate
+          node.copy(predicate: predicate)
+        end
+      end
+    end
+  end
+end