Moses Support Digest:Binarized SRILM

[Moses-support] Binarized SRILM

Hi,
just a quick question, can Moses use a binary version of the srilm lm? I have built it with the option -write-binary-lm

Thanks
Marco


Re:[Moses-support] Binarized SRILM

no-one’s asked this before but i’ll be interested to find out. If you anything more, let me know

Re:[Moses-support] Binarized SRILM

Hi Hieu,
I’ll check… and I’ll let u know.

In case this the output of moses using a binarized srilm LM:
[total=-40.478] <<-8.000, -74.000, 0.000, -11.209, -4.077, -2.675, -12.833, -0.519, -5.449, -312.951, -66.706, -138.406, -42.936, -83.404, 32.997>>
which is the LM value? Is it the 10th value?

Thanks
Marco

Re:[Moses-support] Binarized SRILM

Hi,

if you generate an n-best list, the types of the different feature functions are more transparent, but it is a safe bet that your example “-312.951″ is the unweighted language model weight.

-phi

Re:[Moses-support] Binarized SRILM

Hi Philipp and Hieu,

i run this test:
I build a moses model using 62k training sentences, then I build 2 LMs using the same data (English side):
1)the normal srilm
2)the binarized srilm

then i translate.

I got the exactly the same target sentences (I used the “diff” command to compare them). and Blue score (I did not optimize to speed up the test)

Now I’m running a new test using 1.1 M sentences… I’ll let u know later.

the small LM value in the previous example I guess it is due to small amount of training data (as in that case I used only 62k).

if u have any comment, please let me know

Thanks a lot
Marco

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Related posts:

1. Moses Support Digest:Building POS language model with SRILM
2. Moses Support Digest:SRILM installation problem
3. Moses Support Digest:Hierarchical rule extraction
4. Moses Support Digest: moses-irstlm memory racing with 5-gram lm
5. Moses Support Digest:problem running mosesserver
6. Moses Support Digest: Note that regr. test for ptable-filtering fails
7. Moses Support Digest: cannot find -loolm
8. Moses Support Digest:CreateBerkeleyPt and On-Disk Rule Table
9. Moses Support Digest:moses threads compilation problem with RandLM
10. Moses Support Digest: Moses seems to hang