52nlp's Learning Notes » chart

Moses Support Digest: About mert-moses in mose-chart

52nlp — Tue, 16 Mar 2010 12:12:35 +0000

[Moses-support] About mert-moses in mose-chart

Dear Sir,

When I use mert-moses in Mose-chart with the command:

nohop nice ./mert-moses.pl working-dir/tunin /nc-dev2007.lowercased.fr
working-dir/tuning/lowercased.en
/home/i8001/s0810207/Mose-chartN/mose-chart/moses-cmd/src/moses working-dir/model/moses.ini
–working-dir working-dir/tuning –rootdir /home/i8001/s0810207/Mose-chartN/bin/moses-scripts/scripts-20100302-1240/
–decoder-flags “-v 0″ >& working-dir/tuning/mert.out

I met problem I can’t use hierarchical rules and lexicalised re-ordering at the same time.

I send moses.ini and problem in the attached files

Can you solve this problem for me

A lot of thanks and best regards

Bui Hung Thanh

Re:[Moses-support] About mert-moses in mose-chart

Hi,

lexicalized reordering cannot be used in hierarchical models.

-phi

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Moses Support Digest: About moses-chart

52nlp — Mon, 08 Mar 2010 10:44:46 +0000

[Moses-support] About moses-chart

Dir sir,
When I train rule table for moses chart, at the end of rule I saw two parameter (see rule example)

! ” . [X] ||| ! ” [X] ||| ||| 0.333333 0.004318 1 0.0854249 2.718 ||| 1.5 0.5

Could you please explain the meaning of tow parameter.
Thank you!

–
Thu.

Re:[Moses-support] About moses-chart

hi thuy

the last 2 numbers, 1.5 0.5, are the (fractional) counts used to calculate the probabilities. Its not used for anything at the moment, just for debugging

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Moses Support Digest:mt3_chart compilation error

52nlp — Wed, 03 Mar 2010 16:26:58 +0000

[Moses-support] mt3_chart compilation error

Hi,

When trying to compile mt3_chart i get a compilation error for PhraseDictionaryDynSuffixArray.o, what could be the cause/solution for this, could it be caused by adding -std=c++0x to the AM_CPPFLAGS (ref: http://www.mail-archive.com/moses-support@mit.edu/msg01483.html)? Im compiling with gcc4.4.

g++ -DHAVE_CONFIG_H -I. -I../.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64
-D_LARGE_FILES -std=c++0x -I/home/marce/srilm64/include
-I/home/marce/irstlm/include -g -O2 -MT PhraseDictionaryDynSuffixArray.o
-MD -MP -MF .deps/PhraseDictionaryDynSuffixArray.Tpo -c -o
PhraseDictionaryDynSuffixArray.o PhraseDictionaryDynSuffixArray.cpp
In file included from PhraseDictionaryDynSuffixArray.h:5,
from PhraseDictionaryDynSuffixArray.cpp:1:
DynSuffixArray.h: In member function âvoid
Moses::DynSuffixArray::printAuxArrays()â:
DynSuffixArray.h:43: warning: comparison between signed and unsigned integer
expressions
In file included from PhraseDictionaryDynSuffixArray.cpp:1:
PhraseDictionaryDynSuffixArray.h: In member function âbool
Moses::ScoresComp::operator()(const Moses::Scores&, const Moses::Scores&)
constâ:
PhraseDictionaryDynSuffixArray.h:61: error: âtypeofâ was not declared in
this scope
PhraseDictionaryDynSuffixArray.h:61: error: expected â;â before âitrâ
PhraseDictionaryDynSuffixArray.h:61: error: âitrâ was not declared in this
scope
PhraseDictionaryDynSuffixArray.h:62: error: expected â;â before âitrâ
PhraseDictionaryDynSuffixArray.h:62: error: âitrâ was not declared in this
scope

Thanks,
Marce

Re:[Moses-support] mt3_chart compilation error

maybe. the typeof function is not standard c++ may be causing problems with different compiler versions.

it was used by Abby who is working on suffix arrays. You can try & fix
it, or comment out the code that uses it.

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Moses Support Digest:about the moses-chart reordering

52nlp — Thu, 25 Feb 2010 13:58:17 +0000

[Moses-support] about the moses-chart reordering

Dear all:

Could you tell me where can I find the details of moses-chart reordering? It seems that the reordering table is quite different from that of moses. Is there any papers or documents for it? Thank you very much!

Best regards!

Jie Jiang
CNGL, School of Computing,
Dublin City University,
Glasnevin, Dublin 9.
Tel: +353 (0)1 700 6724

Re:[Moses-support] about the moses-chart reordering

Hi,

the best reference would be David Chiang’s Computational Linguistics article on hierarchical phrase-based models.

-phi

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Moses Support Digest: moses_chart and recaser phrase-table

52nlp — Fri, 22 Jan 2010 14:00:44 +0000

[Moses-support] moses_chart and recaser/phrase-table ?

Hi,

is the moses/moses_chart executable from the mt3_chart branch supposed to be usable as a recaser (that is using a phrase-table) ? When I do so then I get a SEGV (see stacktrace at the very end).

Working on the same files with moses from svn/trunk works fine.

I read somewhere that irstlm is not thread-safe and in the debug output I see that a new thread has been started. Is that the problem? I thought I’d be safe because I did *not* configure moses with –enable-threads nor with boost.

many thanks
Christof

Re:[Moses-support] moses_chart and recaser/phrase-table ?

hi christof

there’s small fiddly changes to the ini file format so the chart decoder isn’t backwardly compatible. You should use the trunk decoder if the data was trained using the trunk scripts

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Moses Support Digest: moses-chart error while compiling training scripts

52nlp — Thu, 21 Jan 2010 13:21:53 +0000

[Moses-support] moses-chart: error while compiling training scripts

I’ve got the following error message when compiling the training scripts from the moses-chart trunk:

g++ tables-core.o extract.o SyntaxTree.o XmlTree.o -o extract
extract.o: In function `printAllHieroPhrases(SentenceAlignment&, int,
int, int, int, HoleCollection&)’:
moses-chart/scripts/training/phrase-extract/extract.cpp:561: undefined
reference to `HoleCollection::SortSourceHoles()’
collect2: ld returned 1 exit status
make[1]: *** [extract] Error 1
make[1]: Leaving directory `moses-chart/scripts/training/phrase-extract’
make: *** [compile] Error 1

Does anyone know what the problem is?
Thanks!

Jörg

Re:[Moses-support] moses-chart: error while compiling training scripts

hi joerg

svn up 1 more time. i forgot to check in the makefile

Re:[Moses-support] moses-chart: error while compiling training scripts

Hi Joerg,

it seems there’s a bug in scripts/training/phrase-extract/Makefile. You need to add “HoleCollection.o” to the compile line and dependencies of “extract”:

extract: tables-core.o [...] HoleCollection.o extract.h
g++ tables-core.o [...] HoleCollection.o -o extract

at least that’s what I did …

best regards
Christof

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Moses Support Digest:compiling moses 3 chart

52nlp — Wed, 09 Dec 2009 12:09:58 +0000

[Moses-support] compiling moses 3 chart

Hello,
I guess this question is for Hugh.
I’m trying to compile the new moses chart parsing decoder in the mt3_chart directory.
I have the berkeleydb libraries under /usr/lib and /usr/include.
I run ./regenerate-makefiles.sh;./configure
–with-srilm=/home/john/srilm –with-berkeleydb=/usr
I get warnings and then an error like:
/usr/include/stdlib.h:351 error expected ; before * token
.
.
.

I have the trunk moses working smoothly. Is there anything extra needed to get the hierarchical chart parsing moses compiled?
Thanks,

–
Regards,
John J Morgan

Re:[Moses-support] compiling moses 3 chart

hi john

I’ve added another phrase table implementation in the chart branch which uses berkeley db so even if you compile trunk ok, you need to do a little extra for this branch. I think if Berkeley db includes & libs are installed in the default locations, you don’t need to specify the –with-berkeleydb option.

Hieu

Re:[Moses-support] compiling moses 3 chart

Hi,

try to compile without the –with-berkeleydb switch and everything should work.

-phi

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.