Atom editor - not receiving the correct scopes from tree sitter parser

Question

I have developed a format called PTL. I would like to have syntax highlighting in Atom for it, so I followed the appropriate guides to create a working tree-sitter parser for it. With a colossal amount of debugging, I managed to get the parser installed as a package that Atom can read.

The language package

The parser package

I tested the parser with the following file: input.ptl:

==================
ptlExample
==================
#define myvar 4
#define myvartwo 2
GlobalSettings
{
    Display
    {
        debugLevel = $(myvar)
        debugLevelzap = 3
    }
    Memory
    {
        allowStackAllocation = false
    }
}
Domain
{
    blockDim = [1, 2, 2]
    coolio = @sqrt($(myvartwo))
}
//parallel
---
(source_file
    (preprocessor_definition
        (preprocessor_name
            (string_identifier)
        )
        (rvalue_statement
            (basic_identifier
                (number_identifier)
            )
        )
    )
    (preprocessor_definition
        (preprocessor_name
            (string_identifier)
        )
        (rvalue_statement
            (basic_identifier
                (number_identifier)
            )
        )
    )
    (section_identifier
        (string_identifier)
        (section_identifier
            (string_identifier)
            (assignment
                (string_identifier)
                (assign_sym)
                (rvalue_statement
                    (variable_invocation
                        (variable_invoke_sym)
                        (rvalue_statement
                            (basic_identifier
                                (string_identifier)
                            )
                        )
                    )
                )
            )
            (assignment
                (string_identifier)
                (assign_sym)
                (rvalue_statement
                    (basic_identifier
                        (number_identifier)
                    )
                )
            )
        )
        (section_identifier
            (string_identifier)
            (assignment
                (string_identifier)
                (assign_sym)
                (rvalue_statement
                    (basic_identifier
                        (boolean_identifier)
                    )
                )
            )
        )
    )
    (section_identifier
        (string_identifier)
        (assignment
            (string_identifier)
            (assign_sym)
            (rvalue_statement
                (vector
                    (rvalue_statement
                        (basic_identifier
                            (number_identifier)
                        )
                    )
                    (rvalue_statement
                        (basic_identifier
                            (number_identifier)
                        )
                    )
                    (rvalue_statement
                        (basic_identifier
                            (number_identifier)
                        )
                    )
                )
            )
        )
        (assignment
            (string_identifier)
            (assign_sym)
            (rvalue_statement
                (function_invocation
                    (function_invocation_sym)
                    (string_identifier)
                    (parameter_list
                        (rvalue_statement
                            (variable_invocation
                                (variable_invoke_sym)
                                (rvalue_statement
                                    (basic_identifier
                                        (string_identifier)
                                    )
                                )
                            )
                        )
                    )
                )
            )
        )
    )
    (comment
        (string_identifier)
    )
)

The test is successful, so I am confident that tree-sitter is capturing the full syntax tree. However, after installation, when I open up my exact file in atom (only the content, with none of the tree-sitter extras like the title and expected syntax tree), nothing is scoped properly, and the whole file has the ptl scope. I have the following scope:

scopes:
  'comment > string_identifier': 'comment.line.double-slash.ptl'

so at the very least, the comment at the end should be picked up. It is not. I have seen this thread describing the exact same issue, but the fix involved updating Atom's grammar fetching function to recognize tree-sitter grammars. I am running a newer version of Atom so this is not the cause. How can I go about debugging this?

EDIT: I have found an error in the console:

Uncaught (in promise) RangeError: Incompatible language version. Compatible range: 9 - 10. Got: 12
    at Parser.setLanguage (/usr/share/atom/reso…sitter/index.js:248)
    at TreeSitterLanguageMode.parse (/usr/share/atom/reso…tatic/<embedded>:11)
    at LanguageLayer._performUpdate (/usr/share/atom/reso…tatic/<embedded>:11)
    at LanguageLayer.update (/usr/share/atom/reso…tatic/<embedded>:11)
    at new TreeSitterLanguageMode (/usr/share/atom/reso…tatic/<embedded>:11)
    at GrammarRegistry.languageModeForGrammarAndBuffer (/usr/share/atom/reso…tatic/<embedded>:11)
    at grammarScoresByBuffer.forEach (/usr/share/atom/reso…tatic/<embedded>:11)
    at Map.forEach (<anonymous>)
    at GrammarRegistry.grammarAddedOrUpdated (/usr/share/atom/reso…tatic/<embedded>:11)
    at GrammarRegistry.addGrammar (/usr/share/atom/reso…tatic/<embedded>:11)
    at TreeSitterGrammar.activate (/usr/share/atom/reso…tatic/<embedded>:11)
    at Package.loadGrammarsSync (/usr/share/atom/reso…tatic/<embedded>:11)
    at Workspace.deserialize (/usr/share/atom/reso…tatic/<embedded>:11)
    at AtomEnvironment.deserialize (/usr/share/atom/reso…static/<embedded>:1)
<embedded>:11 Uncaught (in promise) TypeError: Cannot read property 'rootNode' of null
    at LanguageLayer._populateInjections (/usr/share/atom/reso…tatic/<embedded>:11)
    at e.injectionRegex.currentParsePromise.currentParsePromise.then (/usr/share/atom/reso…tatic/<embedded>:11)

Based on this it looks like I need to rebuild my package with a different version of tree-sitter, though I have no idea which version of tree-sitter Atom uses. Does this mean I need to update Atom's version of tree-sitter (seems preferable), or do I need to downgrade the version of tree-sitter I use in my package?

Answer 1

I ended up upgrading atom to the newest version, then following the same process that I used to get to the state above.