I Love Natural Language Processing

[Moses-support] CALL FOR PAPERS – PBML

CALL FOR PAPERS:
OPEN SOURCE TOOLS FOR MACHINE TRANSLATION
The Fifth Machine Translation Marathon, which will take place September
13-18 in Le Mans, France, is hosting an Open Source Convention to
advance the state of the art in machine translation. The MT Marathon
is organised by the computer science laboratory of Le Mans (LIUM) and the
University of Le Mans on behalf of the EuroMatrixPlus Consortium.
We invite developers of open source tools to present their work and
submit a paper of up to 10 pages that (a) describes the underlying
methodology and (b) includes instructions how to use the tools.
We are looking for stand-alone tools and extensions of existing tools,
such as the Moses open source systems. Accepted papers will be
presented during the MT Marathon and published as a special issue of
the Prague Bulletin of Mathematical Linguistics (

http://ufal.mff.cuni.cz/pbml).

Possible topics:
* training of machine translation models
* machine translation decoders
* tuning of machine translation systems
* evaluation of machine translation
* visualization, annotation or debugging tools
* tools for human translators
* interfaces for web-based services or APIs
* extensions of existing tools
* other tools for machine translation
This is the third time that the MT Marathon will host the Open Source
Convention. The papers from previous years are available online:

http://ufal.mff.cuni.cz/pbml-91-100.html

Papers will be reviewed by two reviewers appointed by the program
committee
e.g. papers which will be marked as the best by the reviewers (with no
substantial
modifications required) will be printed as usual in the PBML journal in time
for the MT Marathon.
Others will be printed after requested modifications in the next PBML
issue.
Important dates:
Deadline for paper submission: August 1 2010
Notification of acceptance: August 7 2010
Camera-ready paper due: August 14 2010
Presentations: September 13-17 2010 (at the MT Marathon in Le Mans)
Camera-ready (next issue) : 15 December 2010
Please send full non-anonymous submissions in PDF to Philipp Koehn <
pkoehn at inf.ed.ac.uk>
and the full Xe(La)TeX source for technical pre-review to Ondřej
Bojar .
Please use the PBML style files from

http://ufal.mff.cuni.cz/pbml-instructions.html

(follow the “short paper” track instructions).
Program Committee
Philipp Koehn
Ondrej Bojar
Holger Schwenk
Loïc Barrault

[Moses-support] moses-irstlm memory racing with 5-gram lm

I’m troubleshooting a new moses system with these components:
1) GIZA++ (SVN rev 8, v 1.0.3)
2) IRSTLM (SVN rev 38, v 5.40.01)
3) Moses (SVN rev 3210, dated 4-26-2010)
4) Ubuntu-server 10.04 LTS 64-bit.
5) 3.4 Ghz Pentium-D with 4gb ram.

Using a 3-gram lm, the system works as expected. Training, tuning and
evaluation a small (135K pairs) en-nl subset of europarl.v5 work fine. BLEU
score was 23.

I then built a 5-gram model, edited the moses.ini config and started
mert-moses-new. It creates a filtered model, and then launches moses. The
memory usage grows and within 10 minutes, the system kills moses.

In both cases, the lm is only the target half of the bitext corpus, about
135K lines.

The moses.ini files:

[lmodel-file]
1 0 3 /media/models/irstlm/europarl.v5.mini/3-gram.nl.blm

[lmodel-file]
1 0 5 /media/models/irstlm/europarl.v5.mini/5-gram.nl.blm

I know of one other who has anyone the same problem with the 4-1-2010
moses build and irstlm from March/April last year.

Any suggestions? Could it be the new Ubuntu or the g++-4.4.1 compiler?

Thanks,
Tom
Read the rest of this entry »

[Moses-support] EMS: table binarization

Hi all,

I’m trying to run a system with a binary phrase table, and it doesn’t appear to be doing the binarization. Looking in experiment.meta the relevant section appears to be

binarize-config
in: config
out: bin-config
pass-unless: binarize-all
rerun-on-change: config
default-name: model/moses.bin.ini
template: $binarize-all $ttable-binarizer $rtable-binarizer OUT IN

I can specify ttable-binarizer and rtable-binarizer, but I can’t work out what the value of binarize-all should be. Any suggestions?

Thanks,
Suzy
Read the rest of this entry »

[Moses-support] processPhraseTable in windows

Dear All,

I would like to know how to run processPhraseTable in windows to make Binary Phrase Tables.

Best Regards,
Kyaw Kyaw Zin
Read the rest of this entry »

[Moses-support] compiling on Mac

I have a doubt about moses.

when I try to compile with make on Mac (Snow Leopard or Leopard), it gives me strange libtool errors (see below). It’s only recently been doing this so might be caused by software update. Its a clean install of OSX so there shouldn’t be resource problems. And it compile fine with xcode.

error:
———————————-
make
make all-recursive
Making all in moses/src
make all-am
/bin/sh ../../libtool –tag=CXX –mode=compile g++ -DHAVE_CONFIG_H -I.
-I../.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES
-DTRACE_ENABLE=1 -g -O2 -MT AlignmentInfo.lo -MD -MP -MF
.deps/AlignmentInfo.Tpo -c -o AlignmentInfo.lo AlignmentInfo.cpp
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: fork: Resource temporarily unavailable
../../libtool: line 712: *** Warning: inferring the mode of operation is
deprecated.: command not found
../../libtool: line 713: *** Future versions of Libtool will require
-mode=MODE be specified.: command not found
../../libtool: line 745: : warning: cannot infer operation mode without
MODE-ARGS: command not found
Read the rest of this entry »

[Moses-support] about CreateOnDiskPt parameter

Hi all:

Could please anyone explain a bit about these two parameter of
CreateOnDiskPt:
[#source factors] [#target factors]
Because when I’m trying to binarize the New format of moses [version
2010/4/26], I have to use
[#source factors] = 1
[#target factors] =1
to make it run successfully.

However, the specification in moses.ini has a factor 0 for both source and
target.

Does it mean in the source code, now moses suppose the source/target words
as the first factor?

Best regards!
Read the rest of this entry »

[Moses-support] experiment management system and Moses scripts

Hi all,

I’ve just downloaded and installed the latest subversion revision (3261) and I’m trying out the new experiment management system, but the Moses scripts are causing me problems. When I point the config variable ‘moses-script-dir’ to ~/mosesdecoder/scripts (the scripts directory inside the repository), executing the ems crashes because BINDIR is not set in train-model.perl. However if I instead run ‘make release’ from ~/mosesdecoder/scripts and set moses-scripts-dir to point to the directory thus created, the ems crashes when checking the config because that scripts directory doesn’t include generic/mteval-v12.pl or the tokenizer and ems directories. Is the solution to update ~/mosesdecoder/scripts/released_files, or to change BINDIR in train-model.perl without running ‘make release’, or something else?

Thanks,
Suzy
Read the rest of this entry »

[Moses-support] Regarding moses.weight-reused.ini

Hello

I have a question to which I have been wanting to know the answer, I
googled, I read but I am still not clear about the following: I replicated
the baseline, following these steps:
1. Preprocessing of training data
2. Actual training ( LM, RM and TM ) =>moses.ini
3. Translating
4. Evaluating
5. Tuning => moses.weight-reused.ini

So far all is ok, but now I need to improve my baseline by trying to find
the suitable parameter values. My question is: when I modify certain
parameters ( like distortion limit, stack size, translation-table etc) do I
use during the translation the moses.ini file or the moses.weigh-reused.
file. I know that the latter has the values for weight-t, weight-l,weight-d,
weight-w different, but I want to “play” with the other parameters as well.
With which one do I work for improvement on the system?

Thank you

Ligia
Read the rest of this entry »

[Moses-support] Moses seems to hang

I just built a new moses system with the newest releases of all
components:
1) GIZA++ (SVN rev 8, v 1.0.3)
2) IRSTLM (SVN rev 38, v 5.40.01)
3) Moses (SVN rev 3210, dated 4-26-2010)

My host system is the new Ubuntu-server 10.04 LTS 64-bit. I’ll be
contributing several patches as soon as I confirm they’re working.

The all components now builds successfully. The LM training goes smoothly,
as does the train-factored-phrase-model.perl training. Binarizing LM,
phrase and reordering tables goes as expected.

When I try to run my first test translations before tuning, Moses seems to
load the reordering table, phrase table and language models. Then, the
system does nothing. There is no output. It stays suspended until I hit
Ctrl-C.

Here’s the input:

dit is niet gebeurd . | moses -f ./model/moses.ini

Here’s the output:

Defined parameters (per moses.ini or switch):
config: ./model/moses.ini
distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6
./model/reordering-table
distortion-limit: 6
input-factors: 0
lmodel-file: 0 0 3 ./lm/irstlm-3.en.blm.mm
mapping: 0 T 0
ttable-file: 0 0 0 5 ./model/phrase-table
ttable-limit: 20
weight-d: 0.3 0.3 0.3 0.3 0.3 0.3 0.3
weight-l: 0.5000
weight-t: 0.2 0.2 0.2 0.2 0.2
weight-w: -1
Loading lexical distortion models…have 1 models
Creating lexical reordering…
weights: 0.300 0.300 0.300 0.300 0.300 0.300
binary file loaded, default OFF_T: -1
Start loading LanguageModel ./lm/irstlm-3.en.blm.mm : [0.000] seconds
Loading Internal LM: ./lm/irstlm-3.en.blm.mm
./lm/europarl.v5.mini-irstlm-3.en.blm.mm
Finished loading LanguageModels : [1.000] seconds
Start loading PhraseTable ./model/phrase-table : [1.000] seconds
filePath: ./model/phrase-table

((( this is where it stops until I hit Ctrl-C )))

I’ve never seen this before. Any suggestions?

Thanks,
Tom
Read the rest of this entry »

[Moses-support] Copyright application issue for a Snippet Translation System using MOSES

Dear Sir,

We have developed a reordered Phrase based SMT System with reordering of input snippets obtained from a search engine. The system translates English / Hindi snippets to the following Indian languages: Hindi/Bengali/Marathi/Tamil/Telugu/Punjabi. For developing this Snippet Translation system we have used MOSES Training Toolkit and Decoder and also the SRILM toolkit for generating the Language model in English .

The above system was developed by us as Member of the Project Team for R&D project entitled: Development of Cross Lingual Information Access System (CLIA) and the machine translation system is being copyrighted with Department of Information Technology (DIT), Government of India, New Delhi & Jadavpur University, Kolkata as the owners.

Please tell us whether we need to include the reference to MOSES in the application for registration of copyright and how we can do that.

Regards,

Sivaji
Read the rest of this entry »