CRFSuite predictions are swallowed if label is ':'?

318 Views Asked by At

I am using CRFSuite for sequence classification (POS tagging). To my surprise it seems like CRFSuite does not like the label':' Units or tokens that have ':' as actual label are entirely skipped (no remark in the prediction output about a missing or skipped item)

I use other punctuation-related labels such as '.' or ',', but these are correctly used and outputted.

Has someone made a similar experience or nows why ':' is skipped ?

1

There are 1 best solutions below

0
On BEST ANSWER

From http://www.chokkan.org/software/crfsuite/tutorial.html:

CRFsuite accepts any string as an attribute name as long as the string does not contain a colon character (that is used to separate an attribute name and its weight).

So if you have an attribute like w[0]=the:0.5, the attribute name is "w[0]=the" and the weight is 0.5.