Anyway to get the TreeSitterTree inside the textView?

[Kaelzs]

Hello, I'm integrating the Runestone framework into my rule editor, and it's awesome, especially the highlight and line number stuffs!

And after integrating the syntax highlight, I try to do some analysis using the tree parsed by the tree-sitter, and I found the tree is stored in textView privately. So, is there any way to get that tree inside the textView?

[simonbs]

@Kaelzs I'm hesitant to expose Tree-sitter through Runestone because by not doing so, we are free to implement new means of parsing the code without modifications to the public API.

However, I also understand that it can be valuable for editors to access some of the information stored in the tree. Runestone is already exposing a bit of information from Tree-sitter through the -syntaxNode(at:) function.

I'm not totally against exposing the Tree-sitter tree in the future but if we can avoid it, I would appreciate that. Can you elaborate on exactly what data you need? Maybe there's a way to expose it without exposing the entire Tree-sitter tree.

[Kaelzs]

I checked -syntaxNode(at:) API, but it might not match my requirement.

I'm writing a framework for some in-app debugging, and I created a config file that follows the format of the INI file, and I need to do some analysis from it, I need to go through the whole tree to generate the config struct by analyzing every type and content from the tree node, and maybe to generate some error information as a log.

I think I need an API to get the iterator from the textView and analyze the nodes one by one, and I need the type, range( or string content), and the position (generate the error message) from every node, and I think It would be great if we could get all these information.

I've checked the information in TreeSitterNode, and I think the relation-related API, such as parent, child(at:) etc is needed. So that after I get the start node (maybe syntaxNode(at: 0) or something elase), I can traverse every child and sibling node from that start node.

[Kaelzs]

I think Runestone can export the TreeSitterNode via some protocols like below

public protocol TSNodeType {
    var type: String? { get }

    var parent: TSNodeType? { get }
    var previousSibling: TSNodeType? { get }
    var nextSibling: TSNodeType? { get }

    var childCount: Int { get }
    var startLocation: TextLocation { get }
    var endLocation: TextLocation { get }

    func child(at index: Int) -> TSNodeType
}

It's similar with the SyntaxNode structure, but provides more information about the relationship, and method to get the root node in textView as well.

protocol InternalLanguageMode {
    var rootSyntaxNode: TSNodeType? { get }
}

extension TextInputView {
    var rootSyntaxNode: TSNodeType? {
        languageMode.rootSyntaxNode
    }
}

and by doing so, the syntax tree can be used to parse the code, and you can still implement new means of parsing the code (I think during the parsing phase, a syntax tree should always been generated).

[simonbs]

Your suggestion makes sense. I'm still a bit hesitant to implement this since it seems quite closely tied to the usage of Tree-sitter. However, I see how this can be useful.

(I think during the parsing phase, a syntax tree should always been generated).

I don't think that's the case when using the popular TextMate grammars which use regular expressions. Originally I hoped that Runestone would support both Tree-sitter and TextMate grammars which is why I'm hesitant to expose any Tree-sitter specifics in the public API.

That said I'm growing more and more hesitant to support TextMate grammars. They're popular and widely available but I'm seeing an increasing interest in moving away from them in favour of more performant solutions like Tree-sitter.

[nighthawk]

I'd also like if there's was a way to access some of the Tree-sitter functionality through the Runestone framework. Tree-sitter is already a bit more than an implementation detail due to TreeSitterLanguage class, so it'd be nice if there was also a way to more out of it.

In my particular use case, my app has a special syntax for defining formulas for analysis data (like formulas in Numbers). I've written my own Tree-sitter grammar for that and am using Runestone as the in-app editor for syntax highlighting. I also have some light-weight autocompletion in that editor, which would benefit a lot from having access to the tree generated for formulas from the same Tree-sitter grammar. Sine Runestone already includes Tree-sitter, it feels weird to have to add another Tree-sitter framework to generate those trees — plus if there are functional differences in the implementations, that could lead to weird behaviour.

In short, I think some more exposure of the Tree-sitter-related functionality could be a good thing for this framework, making Tree-sitter support a richer feature set with some useful output rather than implementation detail and just serving as an input.

Just my 2ct. I'll have a play accessing those trees for my grammar in my fork, and might come back with a useful suggestion from that.

Quick edit: I just noticed TextView.syntaxNode(at:) which already provides a bit of that. That's a very useful starting point.