I Love Natural Language Processing

[Moses-support] About moses-chart

Dir sir,
When I train rule table for moses chart, at the end of rule I saw two parameter (see rule example)

! ” . [X] ||| ! ” [X] ||| ||| 0.333333 0.004318 1 0.0854249 2.718 ||| 1.5 0.5

Could you please explain the meaning of tow parameter.
Thank you!

–
Thu.
Read the rest of this entry »

[Moses-support] mt3_chart compilation error

Hi,

When trying to compile mt3_chart i get a compilation error for PhraseDictionaryDynSuffixArray.o, what could be the cause/solution for this, could it be caused by adding -std=c++0x to the AM_CPPFLAGS (ref: http://www.mail-archive.com/moses-support@mit.edu/msg01483.html)? Im compiling with gcc4.4.

g++ -DHAVE_CONFIG_H -I. -I../.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64
-D_LARGE_FILES -std=c++0x -I/home/marce/srilm64/include
-I/home/marce/irstlm/include -g -O2 -MT PhraseDictionaryDynSuffixArray.o
-MD -MP -MF .deps/PhraseDictionaryDynSuffixArray.Tpo -c -o
PhraseDictionaryDynSuffixArray.o PhraseDictionaryDynSuffixArray.cpp
In file included from PhraseDictionaryDynSuffixArray.h:5,
from PhraseDictionaryDynSuffixArray.cpp:1:
DynSuffixArray.h: In member function âvoid
Moses::DynSuffixArray::printAuxArrays()â:
DynSuffixArray.h:43: warning: comparison between signed and unsigned integer
expressions
In file included from PhraseDictionaryDynSuffixArray.cpp:1:
PhraseDictionaryDynSuffixArray.h: In member function âbool
Moses::ScoresComp::operator()(const Moses::Scores&, const Moses::Scores&)
constâ:
PhraseDictionaryDynSuffixArray.h:61: error: âtypeofâ was not declared in
this scope
PhraseDictionaryDynSuffixArray.h:61: error: expected â;â before âitrâ
PhraseDictionaryDynSuffixArray.h:61: error: âitrâ was not declared in this
scope
PhraseDictionaryDynSuffixArray.h:62: error: expected â;â before âitrâ
PhraseDictionaryDynSuffixArray.h:62: error: âitrâ was not declared in this
scope

Thanks,
Marce
Read the rest of this entry »

[Moses-support] SRILM installation problem

Hi All,

I had a problem to install Moses and posted it here but no response is coming out yet. Can any one help me where to post this. I appreciate you guys for any help.

Mulu G.

Hi All,

I was trying to install srilm on Debian linux 5 lenny, MACHINE_TYPE= i686, with gcc version v 4.3.2. I have changed the Makefile according the manuals and without success. So when I try to enter #make World, it produces the following message

/bin/sh: /root/srilm: is a directory
mkdir include lib bin
mkdir: cannot create directory `include’: File exists
mkdir: cannot create directory `lib’: File exists
mkdir: cannot create directory `bin’: File exists
make: [dirs] Error 1 (ignored)
make init
/bin/sh: /root/srilm: is a directory
make[1]: Entering directory `/root/srilm’
for subdir in misc dstruct lm flm lattice utils; do
(cd $subdir/src; make SRILM=/root/srilm [SRILM] MACHINE_TYPE=
OPTION= MAKE_PIC= init) || exit 1;
done
make[2]: Entering directory `/root/srilm/misc/src’
/root/srilm/common/Makefile.common.variables:96:
/root/srilm/common/Makefile.machine.: No such file or directory
make[2]: *** No rule to make target `/root/srilm/common/Makefile.machine.’.
Stop.
make[2]: Leaving directory `/root/srilm/misc/src’
make[1]: *** [init] Error 1
make[1]: Leaving directory `/root/srilm’
make: *** [World] Error 2

Could you please help me in sorting out the problem and how to fix it.

Thank you all,

Mulu G.
Read the rest of this entry »

[Moses-support] GIZA++ error

sir,

while running the Giza++ by the command

./GIZA++ -S corp.en.vcb -T corp.ta.vcb -C corp.en_corp.ta.snt

we are getting as below :

what is the coocurrence file ?

how to rectify this problem and run Giza++ ?

reading vocabulary files
Source vocabulary list has 35497 unique tokens
Target vocabulary list has 71683 unique tokens
Calculating vocabulary frequencies from corpus corp.en_corp.ta.snt
Reading more sentence pairs into memory …
Corpus fits in memory, corpus has: 14035 sentence pairs.
Train total # sentence pairs (weighted): 14035
Size of source portion of the training corpus: 330148 tokens
Size of the target portion of the training corpus: 262033 tokens
In source portion of the training corpus, only 35496 unique tokens appeared
In target portion of the training corpus, only 71681 unique tokens appeared
lambda for PP calculation in IBM-1,IBM-2,HMM:= 262033/(344183-14035)==
0.793683
ERROR: NO COOCURRENCE FILE GIVEN
Aborted

thank you
Read the rest of this entry »

[Moses-support] Is reordering model a “must-be-used” component to use?

I wonder if I may exclude the reordering model during search.Since I came up with my morpho-syntactical preprocessor to transform source languagein both training-time and run-time, and reordering model deals with the local reordering or wordsduring translation, there doesn’t seem to be the need for using it if the preprocessorcompletely got rid of local/global distortion, even for language model?Is my hypothesis justified?Actually, from my subjective evaluation, using phrase-table and lm alone shows better result thanwhen I use them with reordering model. But I am not sure of my theory.
Read the rest of this entry »

[Moses-support] about the moses-chart reordering

Dear all:

Could you tell me where can I find the details of moses-chart reordering? It seems that the reordering table is quite different from that of moses. Is there any papers or documents for it? Thank you very much!

Best regards!

Jie Jiang
CNGL, School of Computing,
Dublin City University,
Glasnevin, Dublin 9.
Tel: +353 (0)1 700 6724
Read the rest of this entry »

[Moses-support] Dictonary use during training

Hi all, I’m wondering if you would know where I can find an english to spanish parallel, word to word dictionary to complement my training corpus.

Also, from what I have searched I understand you can either add the dictionary words at the end of the corpus or use the giza option. I would like to try both, but for the giza option -d I see that the file format uses the word’s ids, then where will the real words (from the parallel dictionary) go? in the corpus as well? or in a separate file?

Any other suggestions for using a dictionary are welcome.

Thank you.
Read the rest of this entry »

[Moses-support] 2.718 in the phrase-table

The exact purpose of the value 2.718 is what I am not sure of.As far as I know, the value is used for preferring the hypothesis with less phrases tothe one that use more phrases(words) for the same coverage of the source sentence duringthe prefix cost comparison inside priority queue( for the hyps that covers the same source range ).for example ) phrase_1 + phrase_2 + phrase_3 )to prefer hyp1 to hyp2 by multiplying hyp1 by 2.718.This is how I understand the use of the value, since longer phrase has empirically better translationthan the one made up of word-based translation.Is there any one who can confirm my belief, or to correct my conclusion?p.s. Is 2.718 the Euler’s number? If it is, why is the weight determined as such?
Read the rest of this entry »

[Moses-support] mert extractor

Another basic question: When trying to run the mert-moses-new.pl script it seems to require a mert script called extractor, but my training/cmert-0.5/ directory does not contain such a script.

the mert-moses-new.pl contains these two lines:

my $mert_extract_cmd = “$mertdir/extractor”;
my $mert_mert_cmd = “$mertdir/mert”

Where can I find the extractor or what could be the reason it didnt get included, when executing ”make release”.

Thanks!
Marce
Read the rest of this entry »

[Moses-support] Translation from English to Foreign Language

Is it possible to translate with moses from English to any Foreign language?

because I tried using moses, and at “Sanity Check Trained Model” step, the result of best translation always UNK (English-Foreign), but if I translate from Foreign-English, it’s work fine

I modified Train Phrase Model into:
nohup nice
tools/moses-scripts/scripts-YYYYMMDD-HHMM/training/train-factored-phrase-model.perl -scripts-root-dir tools/moses-scripts/scripts-YYYYMMDD-HHMM/ -root-dir
work -corpus work/corpus/news-commentary.lowercased -e en -f id -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
0:3:/home/jschroe1/demo/work/lm/news-commentary.lm >&
work/training.out &

the original was:
nohup nice tools/moses-scripts/scripts-YYYYMMDD-HHMM/training/train-factored-phrase-model.perl -scripts-root-dir tools/moses-scripts/scripts-YYYYMMDD-HHMM/ -root-dir work -corpus work/corpus/news-commentary.lowercased -f fr -e en -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:/home/jschroe1/demo/work/lm/news-commentary.lm >& work/training.out &

I’ve build language model for my foreign language, not English-language-model. Is there anything I missed here?

Laurent
Read the rest of this entry »