An initial test suite for detokenizer.perl.
An initial test suite for detokenizer.perl.

I realize this doesn't quite fit the paradigm if the existing moses test suite.  On the other hand, it's self-contained, easy to run, easy to add tests to (just follow the pattern in the section titled 'Definitions of individual test cases'), and uses an established Perl testing framework.  I don't think it will be infeasible to incorporate it into the existing test suite.

Usage:

run-test-detokenizer.t --results-dir <RESULTS-DIRECTORY>

where <RESULTS-DIRECTORY> is an empty existing directory where the output can be written


git-svn-id: f5d636078e73450abf2183305c5e43efec2a5605@4121 1f5c12ca-751b-0410-a591-d2e778427230

hychyc07/mosesdecoder,hychyc07/mosesdecoder,alvations/mosesdecoder,emjotde/mosesdecoder_nmt,alvations/mosesdecoder,hychyc07/mosesdecoder,pjwilliams/mosesdecoder,hychyc07/mosesdecoder,alvations/mosesdecoder,moses-smt/mosesdecoder,alvations/mosesdecoder,pjwilliams/mosesdecoder,moses-smt/mosesdecoder,tofula/mosesdecoder,emjotde/mosesdecoder_nmt,alvations/mosesdecoder,tofula/mosesdecoder,pjwilliams/mosesdecoder,hychyc07/mosesdecoder,emjotde/mosesdecoder_nmt,KonceptGeek/mosesdecoder,moses-smt/mosesdecoder,pjwilliams/mosesdecoder,emjotde/mosesdecoder_nmt,pjwilliams/mosesdecoder,alvations/mosesdecoder,KonceptGeek/mosesdecoder,moses-smt/mosesdecoder,moses-smt/mosesdecoder,KonceptGeek/mosesdecoder,alvations/mosesdecoder,emjotde/mosesdecoder_nmt,KonceptGeek/mosesdecoder,KonceptGeek/mosesdecoder,hychyc07/mosesdecoder,KonceptGeek/mosesdecoder,tofula/mosesdecoder,hychyc07/mosesdecoder,alvations/mosesdecoder,moses-smt/mosesdecoder,pjwilliams/mosesdecoder,emjotde/mosesdecoder_nmt,tofula/mosesdecoder,tofula/mosesdecoder,tofula/mosesdecoder,moses-smt/mosesdecoder,tofula/mosesdecoder,KonceptGeek/mosesdecoder,tofula/mosesdecoder,tofula/mosesdecoder,alvations/mosesdecoder,hychyc07/mosesdecoder,KonceptGeek/mosesdecoder,pjwilliams/mosesdecoder,emjotde/mosesdecoder_nmt,moses-smt/mosesdecoder,moses-smt/mosesdecoder,moses-smt/mosesdecoder,emjotde/mosesdecoder_nmt,emjotde/mosesdecoder_nmt,pjwilliams/mosesdecoder,alvations/mosesdecoder,pjwilliams/mosesdecoder,pjwilliams/mosesdecoder,hychyc07/mosesdecoder,hychyc07/mosesdecoder,emjotde/mosesdecoder_nmt,tofula/mosesdecoder,KonceptGeek/mosesdecoder,moses-smt/mosesdecoder,alvations/mosesdecoder,KonceptGeek/mosesdecoder,tofula/mosesdecoder