I'm trying to write a lexer that lexes c-style comments and nothing else(for now).
{
module Lexer where
import Prelude hiding (head, take, tail)
import Data.ByteString.Lazy
}
%wrapper "monad-bytestring"
@not_bc_end = ~\* | \* ~\/
tl :-
<0> $white+ ;
<0> "/*" { tok (\p s -> BCBegin p) `andBegin` bc }
<bc> .+ / not_bc_end { tok (\p s -> BCContent p s) }
<bc> "*/" { tok (\p s -> BCEnd p) `andBegin` 0 }
<0> "//" { tok (\p s -> LCBegin p) `andBegin` lc }
<lc> .*$ { tok (\p s -> LCContent p s) }
{
tok :: (AlexPosn -> ByteString -> Token) -> AlexInput -> Int64 -> Alex Token
tok f (p, _, bs, _) len = pure $ f p (take len bs)
data Token
= LCBegin AlexPosn
| LCContent AlexPosn ByteString
| BCBegin AlexPosn
| BCEnd AlexPosn
| BCContent AlexPosn ByteString
| End AlexPosn
alexEOF = pure End
}
The codegen succeeded but the compilation failed with the following error:
templates/wrappers.hs:288:9: error:
• Couldn't match type ‘Token’ with ‘AlexPosn -> Token’
Expected type: Alex (AlexPosn -> Token)
Actual type: Alex Token
• In a stmt of a 'do' block: action (ignorePendingBytes inp__) len
In the expression:
do alexSetInput inp__'
action (ignorePendingBytes inp__) len
In the expression:
let len = n' - n
in
do alexSetInput inp__'
action (ignorePendingBytes inp__) len
The compiler complains that the token type (wrapped in Alex
) have to take an argument of the type AlexPosn
. I got confused because there's already an AlexPosn
in the AlexInput
and the user guide explicitly stated that the token type can bu anything as long as they are kept the same across all the tokens.
Changing the definition of tok
to the following solved the problem:
tok :: (AlexPosn -> ByteString -> Token) -> AlexInput -> Int64 -> Alex (AlexPron -> Token)
tok f (_, _, bs, _) len = pure $ \p -> f p (take len bs)
But I have no idea why is Alex focing the token to take an AlexPosn
I accidentally added a field of type
AlexPosn
to the constructorEnd