Hello,
I am currently trying to train a Moses system, which goes fine until the tuning step. I get the following error :

Creating lexical reordering…
weights: 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111
binary file loaded, default OFF_T: -1
Created lexical orientation reordering
ERROR:Language model type unknown. Probably not compiled
into library
ERROR:no LM created. We probably don’t have it compiled
Exit code: 1
The decoder died. CONFIG WAS -w 0.000000 -lm 0.111111 -d
0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 -tm
0.033333 0.022222 0.033333 0.022222 0.000000

I have checked the archives of the mailing list, and couldn’t find anything to resolve the problem.
The language model line in the run1.moses.ini is the following
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
1 0 5 /home/s0678231/disk1/mt-system/lm/europarl.binlm.1

Is anyone able to help with this ?

Thanks in advance,
Ronan


Re:[Moses-support] Tuning failure with “Language model type unknown.”

Hi Ronan

This is the problem:

> ERROR:Language model type unknown. Probably not compiled
> into library

You need to compile with irstlm in order to use a binarised language model.

See here for instructions:

http://www.statmt.org/moses/?n=Development.GetStarted

regards
Barry

Re:[Moses-support] Tuning failure with “Language model type unknown.”

Hi,
It was resolved by removing the lm-binarizer option in the config file.
Thanks for your answer,

Ronan

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Related posts:

1. Moses Support Digest:tuning tree-based models
2. Moses Support Digest:Building POS language model with SRILM
3. Moses Support Digest: Moses seems to hang
4. Moses Support Digest:ConfusionNet GetSubString error when using lattice with UTF8 input
5. Moses Support Digest: moses-irstlm memory racing with 5-gram lm
6. Moses Support Digest:moses threads compilation problem with RandLM
7. Moses Support Digest:GIZA++ error
8. Moses Support Digest: moses install on 64-bit
9. Moses Support Digest: cannot find -loolm
10. Moses Support Digest:Error compiling on Linux

Hi,
you are correct that for POS LMs the lower order n-gram counts are very different and smoothing is less relevant.
You could train a 7-gram LM with Good Turing smoothing for the lower order n-grams and Kneser-Ney for the higher order n-grams.
I have done this occasionally.

-phi

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Related posts:

1. Moses Support Digest:POS LM
2. Moses Support Digest:Tuning failure with Language model type unknown
3. Moses Support Digest:Binarized SRILM
4. Moses Support Digest:About the hierarchical model of Moses
5. Moses Support Digest:SRILM installation problem
6. Moses Support Digest:Aligned phrase counts
7. Moses Support Digest: different bleu scores from nist and moses scripts
8. Moses Support Digest:Translation from English to Foreign Language
9. Moses Support Digest:Is reordering model a must-be-used component to use?
10. Moses Support Digest: Moses seems to hang