How to consume until a delimiter - with exceptions (Petit Parser for Dart)

Question

Fair enough, previously was a badly worded question. I'm going to take another stab at it.

I'm using Dart's PetitParser to consume input. I want a lexer that will accept any string until one of a number of delimiters is found. The trouble I have is that some of the delimiters are single characters and some are strings, and the characters can be escaped if they're in a single quoted string, and also the delimiters don't count if they're in a single quoted string.

This is my attempted basic setup at the moment:

Parser<String> fpLexer(List<String> delimiters) {
  return delimiters
      .map(string)
      .toChoiceParser()
      .cast<String>()
      .neg()
      .plus();
}

For instance, if my delimiters are " and the word then. And my input string is:

'<a href=\"http://www.w3.org/1999/xhtml\">then</a>' some "Link"

I need a Lexer that will consume up until "Link". The reasoning is that the first set of double quotes are within the single quoted segment AND escaped, and because the word then is also within the quoted segment, neither of those qualifies as a delimiter. So the first delimiter that's reached is the initial " before the word Link.

Hopefully I've explained myself better this time. Thanks for the input!

Answer 1

The examples provided are hard to understand and not minimally reproducible (copy-pastable into the editor), so I am trying to answer the two question I think I can extract.

How to consume only as long as another parser accepts?

There are multiple ways to achieve this: There are the logical predicates and and not. Furthermore, there are the lazy and greedy repeaters that consume a parser until another parser succeeds. In the test-case below the parser consumes digits as long as there are two more digits available after the current one (thus the repetition of 3 times):

final parser = digit().starLazy(digit().repeat(3).not());
expect(parser, isParseSuccess('123', result: ['1'], position: 1));
expect(parser, isParseSuccess('1234', result: ['1', '2'], position: 2));

How to recognize a list of items with optional delimiters?

This is straightforward, the separated-parser creates a list of elements and separators (where the separator can also be an optional parser):

final parser = digit().plusSeparated(char(',').optional());
expect(parser, isParseSuccess('1,2,3', result: isSeparatedList<String, String?>(
    elements: ['1', '2', '3'], separators: [',', ','])));
expect(parser, isParseSuccess('12,3', result: isSeparatedList<String, String?>(
    elements: ['1', '2', '3'], separators: [null, ','])));