Add support for slim files

[omarluq]

I have checked that this feature is not already implemented

• This feature does not exist

Use case

Slim is a fairly popular choice among ruby/rails devs and adding support for it is actually not a lot of work once ERB support is merged

Description

Ruby intellisense for slim printing lines, none-printing lines and ruby code blocks

ruby:
 print_msg = true

- if print_msg
  = 'hello world'.capitalize

Implementation

Following the same Idea in issue#1055 the LSP can require slim gem from the main bundle and provide the feature only if it's found
As for parsing slim, The lsp doesn't have to worry about that!
instead it can use Slim::ERBConverter.new(options).call(slim_code) which outputs erb code

[omarluq]

@vinistock can you assign me this issue?
I would like to take a swing at implementing it once #1184 and Shopify/vscode-ruby-lsp#896 are merged if that's ok!
and do you have any feedbacks on my suggested approach?

[vinistock]

Thank you for the feature suggestion. Does Slim provide a way to extract just the Ruby code while maintaining the correct line and column information?

Ideally, we wouldn't be converting Slim into ERB and then ERB into Ruby, so that we can finally parse the Ruby code. That will be very slow for large files.

That is currently the biggest blocker with ERB support and we can't move forward until we figure that part out.

[omarluq]

The slim gem ships a parses but it doesn't give the line numbers, however it gives a syntax tree in the form of a nested array
so the following example

ruby:
  print_value = true

- if print_value
  p = 'Hello World'.capatlize

would generate this output when put through the slim parser

# [
#   [:multi,
#     [
#       [:slim, :embedded, "ruby",
#         [:multi,
#           [:newline],
#           [:slim, :interpolate, "print_value = true"],
#           [:newline]
#         ]
#       ],
#       [:newline],
#       [:slim, :control, "if print_value",
#         [:multi,
#           [:newline],
#           [:html, :tag, "p", [:html, :attrs],
#             [:slim, :output, true, "'Hello World'.capitalize",
#               [:multi,
#                 [:newline]
#               ]
#             ]
#           ]
#         ]
#       ]
#     ]
#   ]
# ]

Maybe traverse the tree and derive the line numbers and column numbers for the ruby code

[vinistock]

I don't believe that AST will allow using Ruby LSP features. All of our features are based on parsing Ruby with Prism and analyzing the AST.

For that to work properly, we need slim to be able to extract only the Ruby code from templates maintaining their exact line and column positions.

Otherwise, things like semantic highlighting would highlight the wrong range for tokens. Or definition wouldn't be able to find the target based on the position received from the editor.

We would need slim to be able to take a template like this

ruby:
  print_value = true

- if print_value
  p = 'Hello World'.capatlize

And return only the Ruby code in its exact positions

  print_value = true

  if print_value
      'Hello World'.capatlize

Then we can just take that Ruby code, parse it with Prism (which will give us the right line and column information) and all features will just work out of the box.

[omarluq]

@vinistock slim doesn't give us that out of the box, that being said its not really hard to derive a ruby document we just traverse the AST.

require 'slim'
require 'pry'

def traverse_sexp(sexp, indent = 0, is_root = true)
  code = []
  factor = indent.zero? ? 1 : indent
  indent_space = "  " * factor

  sexp.each do |node|
    next unless node.is_a?(Array)

    case node.first
    when :multi
      code << traverse_sexp(node[1..-1], indent, is_root)
    when :slim
      case node[1]
      when :embedded
        code << traverse_sexp(node[3], is_root ? 0 : indent + 1, false)
      when :interpolate, :control, :output
        code << indent_space + node[2].strip
        code << traverse_sexp(node[3], indent + 1, false) if node.length > 3
      end
    when :html
      extra_whitespaces = convert_html_to_whitespaces(node)
      code << indent_space + extra_whitespaces if extra_whitespaces
      if node[4].is_a?(Array) && node[4][0] == :slim
        if [:output, :interpolate].include?(node[4][1])
          index = node[4][1] == :output ? 3 : 2
          code << indent_space + node[4][index].strip + "\n"
        else
          code << traverse_sexp(node[4], indent, false)
        end
      elsif node[4].is_a?(Array) && node[4][0] != :slim
        code << traverse_sexp(node[4], indent, false)
      end
    when :newline
      code << "\n"
    end
  end

  code.join("")
end

def convert_html_to_whitespaces(html_node)
  case html_node[1]
  when :tag
    ' ' * html_node[2].to_s.length
  when :attrs
    html_node[2..-1].reduce(' ') do |whitespaces, attr_node|
      attr_format = attr_node[3][0]
      subindex = attr_format == :static ? 1 : 3
      ' ' * (atattr_nodetr[2].length + attr_node[3][subindex].length)
    end
  end
end

template1 = <<~SLIM
- if items.any?
  table id=items class='table yellow'
  - for item in items
    tr
      td.name = item.name
      td.price = item.price
- else
  p 'No items found.'
SLIM

template2 = <<~SLIM
ruby:
  print_value = true

- if print_value
  p = 'Hello World'.capitalize
SLIM

sexp1 = Slim::Parser.new.call(template1)
sexp2 = Slim::Parser.new.call(template2)

File.write('example1.rb', traverse_sexp(sexp1))
File.write('example2.rb', traverse_sexp(sexp2))

  if items.any?
       
  for item in items
      
          item.name
          item.price
  else
     'No items found.'

print_value = true

if print_value
  'Hello World'.capitalize

[omarluq]

ok so I got a little carried away and made small gem out of this traverse function https://github.com/omarluq/sx-processor

this is for your eyes to review its way more optimized than the traverse function! ideally I would like to add the sx-processor (I'm open to renaming it) class to the ruby-lsp and add a SlimDocument class that uses it to generate the ruby blob for Prism

[vinistock]

Sorry, but if we were to accept a runtime dependency, it would be only implicit (like we do for rubocop) and only for slim itself.

I'm also wondering if this may be a better fit for addons. There are multiple templating engines out there and we probably don't want to maintain support for all of them inside this codebase.

It's not currently possible to do it, but I'm thinking if we might be able to design an API to allow for this.

[omarluq]

@vinistock Apologize for the misunderstanding I didn't mean adding the gem as a runtime dependency I just meant coding the class into the LSP it self, I made the gem for demo proposes, so you can review the approach Im proposing. that being said I do see your point. Im open to both.