Moses Support Digest: EMS table binarization

[Moses-support] EMS: table binarization

Hi all,

I’m trying to run a system with a binary phrase table, and it doesn’t appear to be doing the binarization. Looking in experiment.meta the relevant section appears to be

binarize-config
in: config
out: bin-config
pass-unless: binarize-all
rerun-on-change: config
default-name: model/moses.bin.ini
template: $binarize-all $ttable-binarizer $rtable-binarizer OUT IN

I can specify ttable-binarizer and rtable-binarizer, but I can’t work out what the value of binarize-all should be. Any suggestions?

Thanks,
Suzy
Continue reading

Posted in Moses, SMT | Tagged , , , , , | Leave a comment

Moses Support Digest: processPhraseTable in windows

[Moses-support] processPhraseTable in windows

Dear All,

I would like to know how to run processPhraseTable in windows to make Binary Phrase Tables.

Best Regards,
Kyaw Kyaw Zin
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: compiling on Mac

[Moses-support] compiling on Mac

I have a doubt about moses.

when I try to compile with make on Mac (Snow Leopard or Leopard), it gives me strange libtool errors (see below). It’s only recently been doing this so might be caused by software update. Its a clean install of OSX so there shouldn’t be resource problems. And it compile fine with xcode.

error:
———————————-
make
make all-recursive
Making all in moses/src
make all-am
/bin/sh ../../libtool –tag=CXX –mode=compile g++ -DHAVE_CONFIG_H -I.
-I../.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES
-DTRACE_ENABLE=1 -g -O2 -MT AlignmentInfo.lo -MD -MP -MF
.deps/AlignmentInfo.Tpo -c -o AlignmentInfo.lo AlignmentInfo.cpp
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: line 712: *** Warning: inferring the mode of operation is
deprecated.: command not found
../../libtool: line 713: *** Future versions of Libtool will require
-mode=MODE be specified.: command not found
../../libtool: line 745: : warning: cannot infer operation mode without
MODE-ARGS: command not found
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: about CreateOnDiskPt parameter

[Moses-support] about CreateOnDiskPt parameter

Hi all:

Could please anyone explain a bit about these two parameter of
CreateOnDiskPt:
[#source factors] [#target factors]
Because when I’m trying to binarize the New format of moses [version
2010/4/26], I have to use
[#source factors] = 1
[#target factors] =1
to make it run successfully.

However, the specification in moses.ini has a factor 0 for both source and
target.

Does it mean in the source code, now moses suppose the source/target words
as the first factor?

Best regards!
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: experiment management system and Moses scripts

[Moses-support] experiment management system and Moses scripts

Hi all,

I’ve just downloaded and installed the latest subversion revision (3261) and I’m trying out the new experiment management system, but the Moses scripts are causing me problems. When I point the config variable ‘moses-script-dir’ to ~/mosesdecoder/scripts (the scripts directory inside the repository), executing the ems crashes because BINDIR is not set in train-model.perl. However if I instead run ‘make release’ from ~/mosesdecoder/scripts and set moses-scripts-dir to point to the directory thus created, the ems crashes when checking the config because that scripts directory doesn’t include generic/mteval-v12.pl or the tokenizer and ems directories. Is the solution to update ~/mosesdecoder/scripts/released_files, or to change BINDIR in train-model.perl without running ‘make release’, or something else?

Thanks,
Suzy
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: Regarding moses.weight-reused.ini

[Moses-support] Regarding moses.weight-reused.ini

Hello

I have a question to which I have been wanting to know the answer, I
googled, I read but I am still not clear about the following: I replicated
the baseline, following these steps:
1. Preprocessing of training data
2. Actual training ( LM, RM and TM ) =>moses.ini
3. Translating
4. Evaluating
5. Tuning => moses.weight-reused.ini

So far all is ok, but now I need to improve my baseline by trying to find
the suitable parameter values. My question is: when I modify certain
parameters ( like distortion limit, stack size, translation-table etc) do I
use during the translation the moses.ini file or the moses.weigh-reused.
file. I know that the latter has the values for weight-t, weight-l,weight-d,
weight-w different, but I want to “play” with the other parameters as well.
With which one do I work for improvement on the system?

Thank you

Ligia
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: Moses seems to hang

[Moses-support] Moses seems to hang

I just built a new moses system with the newest releases of all
components:
1) GIZA++ (SVN rev 8, v 1.0.3)
2) IRSTLM (SVN rev 38, v 5.40.01)
3) Moses (SVN rev 3210, dated 4-26-2010)

My host system is the new Ubuntu-server 10.04 LTS 64-bit. I’ll be
contributing several patches as soon as I confirm they’re working.

The all components now builds successfully. The LM training goes smoothly,
as does the train-factored-phrase-model.perl training. Binarizing LM,
phrase and reordering tables goes as expected.

When I try to run my first test translations before tuning, Moses seems to
load the reordering table, phrase table and language models. Then, the
system does nothing. There is no output. It stays suspended until I hit
Ctrl-C.

Here’s the input:

dit is niet gebeurd . | moses -f ./model/moses.ini

Here’s the output:

Defined parameters (per moses.ini or switch):
config: ./model/moses.ini
distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6
./model/reordering-table
distortion-limit: 6
input-factors: 0
lmodel-file: 0 0 3 ./lm/irstlm-3.en.blm.mm
mapping: 0 T 0
ttable-file: 0 0 0 5 ./model/phrase-table
ttable-limit: 20
weight-d: 0.3 0.3 0.3 0.3 0.3 0.3 0.3
weight-l: 0.5000
weight-t: 0.2 0.2 0.2 0.2 0.2
weight-w: -1
Loading lexical distortion models…have 1 models
Creating lexical reordering…
weights: 0.300 0.300 0.300 0.300 0.300 0.300
binary file loaded, default OFF_T: -1
Start loading LanguageModel ./lm/irstlm-3.en.blm.mm : [0.000] seconds
Loading Internal LM: ./lm/irstlm-3.en.blm.mm
./lm/europarl.v5.mini-irstlm-3.en.blm.mm
Finished loading LanguageModels : [1.000] seconds
Start loading PhraseTable ./model/phrase-table : [1.000] seconds
filePath: ./model/phrase-table

((( this is where it stops until I hit Ctrl-C )))

I’ve never seen this before. Any suggestions?

Thanks,
Tom
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: Copyright application issue for a Snippet Translation System using MOSES

[Moses-support] Copyright application issue for a Snippet Translation System using MOSES

Dear Sir,

We have developed a reordered Phrase based SMT System with reordering of input snippets obtained from a search engine. The system translates English / Hindi snippets to the following Indian languages: Hindi/Bengali/Marathi/Tamil/Telugu/Punjabi. For developing this Snippet Translation system we have used MOSES Training Toolkit and Decoder and also the SRILM toolkit for generating the Language model in English .

The above system was developed by us as Member of the Project Team for R&D project entitled: Development of Cross Lingual Information Access System (CLIA) and the machine translation system is being copyrighted with Department of Information Technology (DIT), Government of India, New Delhi & Jadavpur University, Kolkata as the owners.

Please tell us whether we need to include the reference to MOSES in the application for registration of copyright and how we can do that.

Regards,

Sivaji
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: Issues with Score data

[Moses-support] Issues with Score data

Hello

I am trying to tune my system ( baseline ) and I have been having the following error ( copied from my cmert.log file):

Seeding random numbers with system clock
Scorer config string:
Using scorer regularisation strategy: none
Using scorer regularisation window: 0
Using case preservation: 1
Using reference length strategy: closest
Data::score_type BLEU
Data::Scorer type from Scorer: BLEU
BleuScorer: 9
ScoreData: number_of_scores: 9
Loading Data from: statscore.data and features.data
loading feature data from features.data
loading score data from statscore.data
Data loaded : [0] seconds
terminate called after throwing an instance of ‘std::runtime_error’
what(): Score data is empty
/home/user/mtsw/trunk/scripts/training/cmert-0.5/enhanced-mert: line 47:
19558 Aborted $SCRIPTS_ROOTDIR/training/cmert-0.5/mert -d
$size

I don’t understand why it says that the score data is empty.
I have been trying to run my tunning.sh script for the last 3 days, everytime I ended up with errors after (max) 12h running.

Thanks for your help
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment

Moses Support Digest: Note that regr. test for ptable-filtering fails

[Moses-support] Note that regr. test for ptable-filtering fails

Hi,

I just wanted to warn recent committers that the regression test for moses
ptable filtering fails (the built-in filtering, not the script
filter-model-given-input.perl).

Cheers, O.
Continue reading

Posted in Moses, SMT | Tagged , , , | Leave a comment