I'm creating a language using the Intellij IDEA Grammar-Kit plugin, and would like the blocks to be indent-based, rather than brace based. The only way I've found to do this so far is to use the <<indent ...>>
function from https://github.com/kandeshvari/idea-nim/blob/master/src/org/dmitrigb/ideanim/parser/ParserUtil.java.
I have minimized the code from that link to:
package org.intellij.sdk.language;
import com.intellij.lang.LighterASTNode;
import com.intellij.lang.PsiBuilder;
import com.intellij.lang.PsiParser;
import com.intellij.lang.impl.PsiBuilderImpl;
import com.intellij.lang.parser.GeneratedParserUtilBase;
import com.intellij.openapi.util.Key;
import com.intellij.psi.TokenType;
import com.intellij.psi.tree.IElementType;
import com.intellij.psi.tree.TokenSet;
import com.intellij.util.containers.IntIntHashMap;
// https://github.com/kandeshvari/idea-nim/blob/master/src/org/dmitrigb/ideanim/parser/ParserUtil.java
public class SPPParserUtil extends GeneratedParserUtilBase {
private static class ParserState {
enum PrimaryMode {
NORMAL, TYPE_DEF, TYPE_DESC
}
private PsiBuilder builder;
private int currentIndent = 0;
private int pragmaCount = 0;
private PrimaryMode primaryMode = PrimaryMode.NORMAL;
private int semiStmtListCount = 0;
private IntIntHashMap tokIndentCache = new IntIntHashMap();
ParserState(PsiBuilder builder) {
this.builder = builder;
}
private String getPrecedingWhiteSpace() {
int wsOffset = 0;
while (builder.rawLookup(wsOffset - 1) == TokenType.WHITE_SPACE)
--wsOffset;
int wsStart = builder.rawTokenTypeStart(wsOffset);
return builder.getOriginalText().subSequence(wsStart, builder.getCurrentOffset()).toString();
}
int getTokenIndent() {
int tokStart = builder.getCurrentOffset();
if (tokIndentCache.containsKey(tokStart))
return tokIndentCache.get(tokStart);
int indent = -1;
String ws = getPrecedingWhiteSpace();
int nlPos = ws.lastIndexOf('\n');
if (nlPos != -1)
indent = ws.length() - nlPos - 1;
tokIndentCache.put(tokStart, indent);
return indent;
}
}
private static Key<ParserState> parserStateKey = new Key<>("parser-state");
private static ParserState getParserState(PsiBuilder builder) {
return builder.getUserData(parserStateKey);
}
public static boolean indented(PsiBuilder builder, int level, Parser parser) {
ParserState state = getParserState(builder);
int tokIndent = state.getTokenIndent();
if (tokIndent > state.currentIndent) {
int prevIndent = state.currentIndent;
state.currentIndent = tokIndent;
boolean result = parser.parse(builder, level + 1);
state.currentIndent = prevIndent;
return result;
}
return false;
}
}
saved as SPPParserUtil.java
This is my minimal BNF so far:
{
parserUtilClass='org.intellij.sdk.language.SPPParserUtil'
tokens=[
TK_DOT = '.'
TK_ARROW = '->'
TK_STAR = '*'
TK_COLON = ':'
KW_FILE = 'file'
KW_IMPORT = 'import'
TK_EOL = 'regexp:[\n|\r]*'
TK_IDENTIFIERS = 'regexp:[a-zA-Z_][a-zA-Z0-9_]*(,\s+[a-zA-Z_][a-zA-Z0-9_]*)*'
TK_SCOPED_IDENTIFIER = 'regexp:[a-zA-Z_][a-zA-Z0-9_]*(\.[a-zA-Z_][a-zA-Z0-9_]*)*'
TK_SPACE = 'regexp:\s+'
]
}
root ::= Program
Program ::=
ProgramContents <<eof>>
ProgramContents ::=
FileDefinition ImportBlock? TK_EOL
FileDefinition ::=
KW_FILE TK_SPACE ScopedIdentifier TK_EOL
ImportLocation ::=
TK_DOT* ScopedIdentifier
ImportDefinition ::=
ImportLocation TK_SPACE TK_ARROW TK_SPACE (Identifiers | TK_STAR) TK_EOL
ImportBlock ::=
KW_IMPORT TK_COLON <<indented ImportDefinition+>>
ScopedIdentifier ::=
TK_SCOPED_IDENTIFIER
Identifiers ::=
TK_IDENTIFIERS
Here is some example code that should pass:
file src.classes.class1
import:
src.classes.class2 -> class2, e2
src.classes.class3 -> class3, e3
src.classes.class4 -> *
However, whenever I try to use <<indent ...>>
in my BNF there is always an error: The error described is ':' expected, got ':'
, on the colon following "import". Basically whatever is before the <<indent ...>>
seems to be consumed by the <<indent ...>>
. How would I change the BNF / use of the ParseUtil
to fix this?
EDIT
I changed the indented
function to always return true, and not use any of the other Java code and the error persists; this implies that the issue isn't with the Java code but maybe how I'm calling it in the BNF file?