I Love Natural Language Processing

[Moses-support] Hierarchical rule extraction

Hieu / Philipp,
I’d like to extract a hierarchical grammar using the Moses tools (which Philipp helpfully pointed out have some documentation at http://www.statmt.org/moses/?n=Moses.ChartDecoding)
Here’s the catch: I already have run train-factored-phrase-model.perl to extract a regular Moses phrase table using trunk. That process already ran GIZA++, and I’d really rather not have to re-run alignment over the exact same data set just to get the branched version of train-factored-phrase-model.perl to let me pass the additional -hierarchical-glue-grammar flags.
Is there a way to run the branch version of train-factored-phrase-model.perl with -hierarchical -glue-grammar, and have it re-use the existing alignments that were created by my previous run of (trunk) train-factored-phrase-model.perl?
Cheers,
Lane


Re:[Moses-support] Hierarchical rule extraction
Hi Lane

this might be what you’re after

1. extract:

/exports/home/s0565741/inf_iccs_smt/hieu/bin/scripts-20091006-1453/training/train-factored-phrase-model.perl
-dont-zip -first-step 5 -last-step 5 -scripts-root-dir
/exports/home/s0565741/inf_iccs_smt/hieu/bin/scripts-20091006-1453 -f de
-e en -alignment grow-diag-final-and -max-phrase-length 5 -hierarchical
-glue-grammar -alignment-file
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/aligned.1
-alignment-stem
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/aligned.1
-extract-file
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/extract.1
-corpus
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/training/corpus.1
-glue-grammar-file
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/glue-grammar.1

2. build table

/exports/home/s0565741/inf_iccs_smt/hieu/bin/scripts-20091006-1453/training/train-factored-phrase-model.perl
-dont-zip -first-step 6 -last-step 6 -scripts-root-dir
/exports/home/s0565741/inf_iccs_smt/hieu/bin/scripts-20091006-1453 -f de
-e en -alignment grow-diag-final-and -max-phrase-length 5 -hierarchical
-glue-grammar -extract-file
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/extract.1
-lexical-file
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/lex.1
-phrase-translation-table
/exports/home/s0565741/inf_iccs_smt/hieu/exp/de-en/nc/working-dir/model/phrase-table.1

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Related posts:

1. Moses Support Digest:About the hierarchical model of Moses
2. Moses Support Digest: Hierarchical and syntax-based decoding in Moses
3. Moses Support Digest:CreateBerkeleyPt and On-Disk Rule Table
4. Moses Support Digest:Aligned phrase counts
5. Moses Support Digest:Moses Error in training phrase
6. Moses Support Digest: About mert-moses in mose-chart
7. Moses Support Digest:Translation from English to Foreign Language
8. Moses Support Digest:About giza++ options when running moses
9. Moses Support Digest:compiling moses 3 chart
10. Moses Support Digest: moses-chart error while compiling training scripts