I Love Natural Language Processing

[Moses-support] Is reordering model a “must-be-used” component to use?

I wonder if I may exclude the reordering model during search.Since I came up with my morpho-syntactical preprocessor to transform source languagein both training-time and run-time, and reordering model deals with the local reordering or wordsduring translation, there doesn’t seem to be the need for using it if the preprocessorcompletely got rid of local/global distortion, even for language model?Is my hypothesis justified?Actually, from my subjective evaluation, using phrase-table and lm alone shows better result thanwhen I use them with reordering model. But I am not sure of my theory.

Re:[Moses-support] Is reordering model a “must-be-used” component to use?

The reordering model tries to deal with global reordering rather than local reordering because the latter is generally solved with phrase generation.

So it depends on the language pair used. Cases like Catalan-Spanish may omit the use of reordering models because they are very much alike. English-Spanish or Chinese-English, on the other hand, consider the reordering model as a “must-be-used” component. Phrase generation will not be able to deal with reordering alone there.

You may read more on the Lexicalized Reordering Model here

http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc1

or you may got to the Tillmann paper here

http://portal.acm.org/citation.cfm?id=1613984.1614010

Which languages are you working on?

Re:[Moses-support] Is reordering model a “must-be-used” component to use?

Do you mean “reordering model” in general? If so, it must be used unless the preprocessor “completely got rid of local/global distortion”.

On the other hand, some specific reordering models such as Lexicalized Rerordering Model in Moses may not be useful. Distance-based reordering doesn’t deal with global reordering well, rather it relocate some phrases within a certain boundary.

Despite many researches have been suggested for global reordering, it is a still unsolved problem..

Re:[Moses-support] Is reordering model a “must-be-used” component to use?

I mean the model for the local distortion event when only surface(token) factor is considered, with maximum length 3. I’ve been experimenting with quasi-syntax-based approach in which I transform source corpus ( English ) in a way that (hopefully) minimizes both local and global distortion at training/run-time, using transfer rules I’ve empirically but manually induced.

Yes, the preprocessor is not perfect in getting rid of global distortion, but it works quite well.

Currently I am working on better way to train the “preprocessing” model, not dependent on a rule-based preprocessing step.

NOTICE:This is digested from the Moses-support mailing list, which supports for the moses SMT decoder.

Related posts:

1. Moses Support Digest:about the moses-chart reordering
2. Moses Support Digest:ConfusionNet GetSubString error when using lattice with UTF8 input
3. Moses Support Digest: moses-irstlm memory racing with 5-gram lm
4. Moses Support Digest: Moses seems to hang
5. Moses Support Digest:Translation from English to Foreign Language
6. Moses Support Digest:problem running mosesserver
7. Moses Support Digest:Tuning failure with Language model type unknown
8. Moses Support Digest: About mert-moses in mose-chart
9. Moses Support Digest:Building POS language model with SRILM
10. Moses Support Digest: Copyright application issue for a Snippet Translation System using MOSES