Difference between registerDocumentSemanticTokensProvider and setMonarchTokensProvider in Monaco Editor?

Question

I am new to Monaco Editor, and i found in the official website, if you want to register your custom semantic token highlight, you can do two ways: using the native method registerDocumentSemanticTokensProvider or using the setMonarchTokensProvider provided by Monarch.

So i am wondering is there any difference between these two methods, and in general, which one is better or in other words which one should i use to provide a language's semantic token?

Answer 1

The API setMonarchTokensProvider takes an interface which describes how to tokenize the input (much like what a lexer does in a usual parser/lexer setup, but in a declarative manner, using regular expressions).

Semantic tokens are a step above this, as they describe semantic (additional meaning) for a (lexer) token. As an example: a lexer (or that Monarch token provider) classify input as tokens of type number, string, id etc. The semantic tokens provider can take ids and determine if they actually represent classes, variables etc.