Error at ARPA model training with SRILM

Question

I have followed this tutorial.

After I run this code:

ngram-count -kndiscount -interpolate -text train-text.txt -lm your.lm

It gives me this error:

"One of modified KneserNey discounts is negative error in discount estimator for order 2."

How can I solve this problem?

Answer 1

There are 2 possibilities that -kndiscount is giving you an error:

1. Your dataset is small.
2. Your phrases in the dataset are small which is a hurdle for a proper ngram training using kndiscount.

The solution could be:

1. Create a bigger dataset.
2. Use other discount techniques.

For reference have a look at this thread: https://sourceforge.net/p/cmusphinx/discussion/help/thread/97ae0e5c/#a760