I Love Natural Language Processing

[Moses-support] How do you solve this moses problem? Thanks

Hi friend,

I have encountered the same problem as somebody, like in this link:

http://www.mail-archive.com/moses-support@mit.edu/msg01646.html

Before three months, I did the step-by-step guide for moses in

http://www.statmt.org/moses_steps.html

Then, everything is right.

I do not know how to solve this problem. I want to know whether you have solved it. If so, could you please tell me how to manage it?

Best wishes;
Jun Lang


Re:[Moses-support] How do you solve this moses problem? Thanks

Hi!

Yes, I managed to solve it.

During one my earlier attempts Giza++ had generated the vocabulary
incorrectly for some reason.
When the training perl script ran again it did not create the giza++ output files again as they existed earlier.
This was the problem in my case – the faulty files that were being used.

I realised that you should always clean up your work directory if you face an error as it can lead to some inconsistencies.
Delete all files generated by Giza++/Moses and try again.

I also removed some trailing non-significant terms from the vocab file
(terms with very low frequency such as numeric values, symbols, incorrect spellings etc that creep in as part of the vocabulary).
Dont think this would cause the problem as it would affect translations more than anything else but I think its a good idea to do so nevertheless.

Ensure your training data is clean, sentences are aligned and lower cased(if applicable). Hope this helps.

Regards,
Danish

Re:[Moses-support] How do you solve this moses problem? Thanks

Hi,

I met the same problem recently and found the problem caused by perl interpreter in the ~~/training/symal/giza2bal.pl which is “/usr/bin/perl^M”, you can use “s/s+$//” to clean the ^M in the first line and maybe this could help you.

Best wishes
Gary

Re:[Moses-support] How do you solve this moses problem? Thanks

Also clean the giza2bal.pl files
You can run dos2unix command to get rid of the ^M characters.

Re:[Moses-support] How do you solve this moses problem? Thanks

Thank you all very much.

Just now, I managed this problem. My original running environment is only 4G ram. When I changed to another environment with 16G ram, the running is right.

Best wishes;
Jun Lang

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Related posts:

1. Moses Support Digest:GIZA++ error
2. Moses Support Digest:About giza++ options when running moses
3. Moses Support Digest:Moses Error in training phrase
4. Moses Support Digest:About the hierarchical model of Moses
5. Moses Support Digest:dictionary problem solved
6. Moses Support Digest:running giza in parts
7. Moses Support Digest:Moses liscencing terms when used in a commercial product
8. Moses Support Digest:How to run giza++ with a dictionary
9. Moses Support Digest:Hierarchical rule extraction
10. Moses Support Digest:Moses step 1 – data preparation step