From c2a5599787ce89385210a9fafb050aed6c7f48f5 Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Tue, 12 Apr 2022 21:43:28 -0400 Subject: [PATCH 1/8] Bump to version 2.1.0 --- CHANGELOG.md | 18 +++++++++++++++++- Gemfile.lock | 2 +- lib/syntax_tree/version.rb | 2 +- 3 files changed, 19 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index afeb8899..189509c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a ## [Unreleased] +## [2.1.0] - 2022-04-12 + +### Added + +- The `SyntaxTree::Visitor` class now implements the visitor pattern for Ruby nodes. +- The `SyntaxTree::Visitor.visit_method(name)` method. +- Support for Ruby 2.7. +- Support for comments on `rescue` and `else` keywords. +- `SyntaxTree::Location` now additionally has `start_column` and `end_column`. +- The CLI now accepts content over STDIN for the `ast`, `check`, `debug`, `doc`, `format`, and `write` commands. + +### Removed + +- The missing hash value inlay hints have been removed. + ## [2.0.1] - 2022-03-31 ### Changed @@ -128,7 +143,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a - 🎉 Initial release! 🎉 -[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.0.1...HEAD +[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.1.0...HEAD +[2.1.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.0.1...v2.1.0 [2.0.1]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.0.0...v2.0.1 [2.0.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v1.2.0...v2.0.0 [1.2.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v1.1.1...v1.2.0 diff --git a/Gemfile.lock b/Gemfile.lock index 989aeb21..e32b11e1 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,7 +1,7 @@ PATH remote: . specs: - syntax_tree (2.0.1) + syntax_tree (2.1.0) GEM remote: https://rubygems.org/ diff --git a/lib/syntax_tree/version.rb b/lib/syntax_tree/version.rb index fde7d633..f3bc7b04 100644 --- a/lib/syntax_tree/version.rb +++ b/lib/syntax_tree/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module SyntaxTree - VERSION = "2.0.1" + VERSION = "2.1.0" end From 9598362201f071627d53e16cf020b9efbdb32d43 Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Tue, 12 Apr 2022 21:52:49 -0400 Subject: [PATCH 2/8] Document plugins --- README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/README.md b/README.md index 32c13b77..384cc605 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ It is built with only standard library dependencies. It additionally ships with - [textDocument/formatting](#textdocumentformatting) - [textDocument/inlayHints](#textdocumentinlayhints) - [syntaxTree/visualizing](#syntaxtreevisualizing) +- [Plugins](#plugins) - [Contributing](#contributing) - [License](#license) @@ -307,6 +308,26 @@ Implicity, the `2 * 3` is going to be executed first because the `*` operator ha The language server additionally includes this custom request to return a textual representation of the syntax tree underlying the source code of a file. Language server clients can use this to (for example) open an additional tab with this information displayed. +## Plugins + +You can register additional languages that can flow through the same CLI with Syntax Tree's plugin system. To register a new language, call: + +```ruby +SyntaxTree.register_handler(".mylang", MyLanguage) +``` + +In this case, whenever the CLI encounters a filepath that ends with the given extension, it will invoke methods on `MyLanguage` instead of `SyntaxTree` itself. To make sure your object conforms to each of the necessary APIs, it should implement: + +* `MyLanguage.read(filepath)` - usually this is just an alias to `File.read(filepath)`, but if you need anything else that hook is here. +* `MyLanguage.parse(source)` - this should return the syntax tree corresponding to the given source. Those objects should implement the `pretty_print` interface. +* `MyLanguage.format(source)` - this should return the formatted version of the given source. + +Below are listed all of the "official" plugins hosted under the same GitHub organization, which can be used as references for how to implement other plugins. + +* [SyntaxTree::Haml](https://github.com/ruby-syntax-tree/syntax_tree-haml) for the [Haml template language](https://haml.info/). +* [SyntaxTree::JSON](https://github.com/ruby-syntax-tree/syntax_tree-json) for JSON. +* [SyntaxTree::RBS](https://github.com/ruby-syntax-tree/syntax_tree-rbs) for the [RBS type language](https://github.com/ruby/rbs). + ## Contributing Bug reports and pull requests are welcome on GitHub at https://github.com/ruby-syntax-tree/syntax_tree. From 2dde243a65de51bddf505c14e9d57c8724610380 Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Wed, 13 Apr 2022 15:45:48 -0400 Subject: [PATCH 3/8] Block ampersand association Fix parsing expressions like `foo.instance_exec(&T.must(block))`, where there are two `args_add_block` calls with a single `&`. Previously it was associating the `&` with the wrong block. --- CHANGELOG.md | 4 ++++ lib/syntax_tree/parser.rb | 20 ++++++++++---------- test/fixtures/arg_block.rb | 2 ++ 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 189509c0..1c793645 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a ## [Unreleased] +### Changed + +- [#45](https://github.com/ruby-syntax-tree/syntax_tree/issues/45) - Fix parsing expressions like `foo.instance_exec(&T.must(block))`, where there are two `args_add_block` calls with a single `&`. Previously it was associating the `&` with the wrong block. + ## [2.1.0] - 2022-04-12 ### Added diff --git a/lib/syntax_tree/parser.rb b/lib/syntax_tree/parser.rb index 60923b57..77660f48 100644 --- a/lib/syntax_tree/parser.rb +++ b/lib/syntax_tree/parser.rb @@ -414,12 +414,19 @@ def on_args_add(arguments, argument) # (false | untyped) block # ) -> Args def on_args_add_block(arguments, block) + # First, see if there is an & operator that could potentially be + # associated with the block part of this args_add_block. If there is not, + # then just return the arguments. operator = find_token(Op, "&", consume: false) - - # If we can't find the & operator, then there's no block to add to the - # list, so we're just going to return the arguments as-is. return arguments unless operator + # If there are any arguments and the operator we found from the list is + # not after them, then we're going to return the arguments as-is because + # we're looking at an & that occurs before the arguments are done. + if arguments.parts.any? && operator.location.start_char < arguments.location.end_char + return arguments + end + # Now we know we have an & operator, so we're going to delete it from the # list of tokens to make sure it doesn't get confused with anything else. tokens.delete(operator) @@ -428,13 +435,6 @@ def on_args_add_block(arguments, block) location = operator.location location = operator.location.to(block.location) if block - # If there are any arguments and the operator we found from the list is - # not after them, then we're going to return the arguments as-is because - # we're looking at an & that occurs before the arguments are done. - if arguments.parts.any? && location.start_char < arguments.location.end_char - return arguments - end - # Otherwise, we're looking at an actual block argument (with or without a # block, which could be missing because it could be a bare & since 3.1.0). arg_block = ArgBlock.new(value: block, location: location) diff --git a/test/fixtures/arg_block.rb b/test/fixtures/arg_block.rb index d423efa8..3e8524ce 100644 --- a/test/fixtures/arg_block.rb +++ b/test/fixtures/arg_block.rb @@ -18,3 +18,5 @@ def foo(&) bar(&) end +% # https://github.com/ruby-syntax-tree/syntax_tree/issues/45 +foo.instance_exec(&T.must(block)) From 31f6e313b623ad4939f86828b7a064febadc48b8 Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Thu, 14 Apr 2022 21:50:05 -0400 Subject: [PATCH 4/8] Handle not() --- lib/syntax_tree/node.rb | 4 ++-- lib/syntax_tree/parser.rb | 10 ++++------ test/fixtures/not.rb | 6 ++++++ 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/lib/syntax_tree/node.rb b/lib/syntax_tree/node.rb index fa4e0829..9e96bd96 100644 --- a/lib/syntax_tree/node.rb +++ b/lib/syntax_tree/node.rb @@ -8168,7 +8168,7 @@ def deconstruct_keys(keys) # not value # class Not < Node - # [untyped] the statement on which to operate + # [nil | untyped] the statement on which to operate attr_reader :statement # [boolean] whether or not parentheses were used @@ -8205,7 +8205,7 @@ def deconstruct_keys(keys) def format(q) q.text(parentheses ? "not(" : "not ") - q.format(statement) + q.format(statement) if statement q.text(")") if parentheses end end diff --git a/lib/syntax_tree/parser.rb b/lib/syntax_tree/parser.rb index 77660f48..d5b82507 100644 --- a/lib/syntax_tree/parser.rb +++ b/lib/syntax_tree/parser.rb @@ -2837,19 +2837,17 @@ def on_unary(operator, statement) # parentheses they don't get reported as a paren node for some reason. beginning = find_token(Kw, "not") - ending = statement + ending = statement || beginning + parentheses = source[beginning.location.end_char] == "(" - range = beginning.location.end_char...statement.location.start_char - paren = source[range].include?("(") - - if paren + if parentheses find_token(LParen) ending = find_token(RParen) end Not.new( statement: statement, - parentheses: paren, + parentheses: parentheses, location: beginning.location.to(ending.location) ) else diff --git a/test/fixtures/not.rb b/test/fixtures/not.rb index eaa456f1..0204e345 100644 --- a/test/fixtures/not.rb +++ b/test/fixtures/not.rb @@ -1,4 +1,10 @@ % +not() +% +not () +% not foo % not(foo) +% +not (foo) From 3459901665584cb9d3d50adbbd278b819b1aa13a Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Sat, 16 Apr 2022 12:58:31 -0400 Subject: [PATCH 5/8] foo::(1) --- lib/syntax_tree/parser.rb | 10 ++++++++-- test/fixtures/call.rb | 14 ++++++++++++++ 2 files changed, 22 insertions(+), 2 deletions(-) create mode 100644 test/fixtures/call.rb diff --git a/lib/syntax_tree/parser.rb b/lib/syntax_tree/parser.rb index d5b82507..92a58ccb 100644 --- a/lib/syntax_tree/parser.rb +++ b/lib/syntax_tree/parser.rb @@ -761,8 +761,14 @@ def on_break(arguments) # (:call | Backtick | Const | Ident | Op) message # ) -> Call def on_call(receiver, operator, message) - ending = message - ending = operator if message == :call + ending = + if message != :call + message + elsif operator != :"::" + operator + else + receiver + end Call.new( receiver: receiver, diff --git a/test/fixtures/call.rb b/test/fixtures/call.rb new file mode 100644 index 00000000..874d290c --- /dev/null +++ b/test/fixtures/call.rb @@ -0,0 +1,14 @@ +% +foo.bar +% +foo.() +% +foo::() +- +foo.() +% +foo.(1) +% +foo::(1) +- +foo.(1) From 555e77b47048edceb63d5b0ecacda26f6846e118 Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Sat, 16 Apr 2022 13:07:34 -0400 Subject: [PATCH 6/8] Empty HshPtn --- lib/syntax_tree/node.rb | 6 ++++-- lib/syntax_tree/parser.rb | 12 +++++++++--- test/fixtures/hshptn.rb | 4 ++++ 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/lib/syntax_tree/node.rb b/lib/syntax_tree/node.rb index 9e96bd96..afe50e4e 100644 --- a/lib/syntax_tree/node.rb +++ b/lib/syntax_tree/node.rb @@ -4536,6 +4536,7 @@ def deconstruct_keys(keys) def format(q) parts = keywords.map { |(key, value)| KeywordFormatter.new(key, value) } parts << KeywordRestFormatter.new(keyword_rest) if keyword_rest + contents = -> do q.seplist(parts) { |part| q.format(part, stackable: false) } end @@ -4546,8 +4547,9 @@ def format(q) return end - parent = q.parent - if PATTERNS.include?(parent.class) + if parts.empty? + q.text("{}") + elsif PATTERNS.include?(q.parent.class) q.text("{ ") contents.call q.text(" }") diff --git a/lib/syntax_tree/parser.rb b/lib/syntax_tree/parser.rb index 92a58ccb..c99eea4b 100644 --- a/lib/syntax_tree/parser.rb +++ b/lib/syntax_tree/parser.rb @@ -1501,13 +1501,19 @@ def on_heredoc_end(value) # (nil | VarField) keyword_rest # ) -> HshPtn def on_hshptn(constant, keywords, keyword_rest) - parts = [constant, keywords, keyword_rest].flatten(2).compact + parts = [constant, *keywords&.flatten(1), keyword_rest].compact + location = + if parts.empty? + find_token(LBrace).location.to(find_token(RBrace).location) + else + parts[0].location.to(parts[-1].location) + end HshPtn.new( constant: constant, - keywords: keywords, + keywords: keywords || [], keyword_rest: keyword_rest, - location: parts[0].location.to(parts[-1].location) + location: location ) end diff --git a/test/fixtures/hshptn.rb b/test/fixtures/hshptn.rb index 2efe2fd3..857336d4 100644 --- a/test/fixtures/hshptn.rb +++ b/test/fixtures/hshptn.rb @@ -44,3 +44,7 @@ case foo in Foo[**bar] end +% +case foo +in {} +end From b4de758ede0efb99502ebda99566eb758143a6d0 Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Sat, 16 Apr 2022 13:18:04 -0400 Subject: [PATCH 7/8] in **nil --- lib/syntax_tree/node.rb | 6 +++++- lib/syntax_tree/parser.rb | 2 +- test/fixtures/hshptn.rb | 4 ++++ 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/syntax_tree/node.rb b/lib/syntax_tree/node.rb index afe50e4e..e43eeae1 100644 --- a/lib/syntax_tree/node.rb +++ b/lib/syntax_tree/node.rb @@ -8677,7 +8677,11 @@ def deconstruct_keys(keys) end def format(q) - q.format(value) if value + if value == :nil + q.text("nil") + elsif value + q.format(value) + end end end diff --git a/lib/syntax_tree/parser.rb b/lib/syntax_tree/parser.rb index c99eea4b..38ea4eef 100644 --- a/lib/syntax_tree/parser.rb +++ b/lib/syntax_tree/parser.rb @@ -2991,7 +2991,7 @@ def on_var_alias(left, right) # ) -> VarField def on_var_field(value) location = - if value + if value && value != :nil value.location else # You can hit this pattern if you're assigning to a splat using diff --git a/test/fixtures/hshptn.rb b/test/fixtures/hshptn.rb index 857336d4..8220e72f 100644 --- a/test/fixtures/hshptn.rb +++ b/test/fixtures/hshptn.rb @@ -48,3 +48,7 @@ case foo in {} end +% +case foo +in **nil +end From f8abd8e5598c788752981d9ceac459237080678b Mon Sep 17 00:00:00 2001 From: Kevin Newton Date: Sat, 16 Apr 2022 21:20:05 -0400 Subject: [PATCH 8/8] Bump to v2.1.1 --- CHANGELOG.md | 9 ++++++++- Gemfile.lock | 2 +- lib/syntax_tree/version.rb | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c793645..b99b3bfb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,9 +6,15 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a ## [Unreleased] +## [2.1.1] - 2022-04-16 + ### Changed - [#45](https://github.com/ruby-syntax-tree/syntax_tree/issues/45) - Fix parsing expressions like `foo.instance_exec(&T.must(block))`, where there are two `args_add_block` calls with a single `&`. Previously it was associating the `&` with the wrong block. +- [#47](https://github.com/ruby-syntax-tree/syntax_tree/pull/47) - Handle expressions like `not()`. +- [#48](https://github.com/ruby-syntax-tree/syntax_tree/pull/48) - Handle special call syntax with `::` operator. +- [#49](https://github.com/ruby-syntax-tree/syntax_tree/pull/49) - Handle expressions like `case foo; in {}; end`. +- [#50](https://github.com/ruby-syntax-tree/syntax_tree/pull/50) - Parsing expressions like `case foo; in **nil; end`. ## [2.1.0] - 2022-04-12 @@ -147,7 +153,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a - 🎉 Initial release! 🎉 -[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.1.0...HEAD +[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.1.1...HEAD +[2.1.1]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.1.0...v2.1.1 [2.1.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.0.1...v2.1.0 [2.0.1]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v2.0.0...v2.0.1 [2.0.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v1.2.0...v2.0.0 diff --git a/Gemfile.lock b/Gemfile.lock index e32b11e1..0fd9de74 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,7 +1,7 @@ PATH remote: . specs: - syntax_tree (2.1.0) + syntax_tree (2.1.1) GEM remote: https://rubygems.org/ diff --git a/lib/syntax_tree/version.rb b/lib/syntax_tree/version.rb index f3bc7b04..aed99b33 100644 --- a/lib/syntax_tree/version.rb +++ b/lib/syntax_tree/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module SyntaxTree - VERSION = "2.1.0" + VERSION = "2.1.1" end