Word Error Rate
By aligning a system's output to a human reference translation, insertions, deletions and substitutions of words can be assessed.
Edit Rate Metrics is the main subject of 12 publications.
Word error rate was first used for the evaluation of statistical machine translation by Tillmann et al. (1997)
, who also introduce position-independent error rate. Allowing block movement (Leusch et al., 2003)
leads to the definition of the CDER metric (Leusch et al., 2006)
. TER allows for arbitrary block movements (Snover et al., 2006)
. MaxSIM uses an efficient polynomial matching algorithm that also uses lemma and part-of-speech tag matches (Chan and Ng, 2008)
The way automatic evaluation metrics work also depends on the tokenization of system output and reference translations (Leusch et al., 2005)
- Popović (2016)
- Wang et al. (2016)
- Dreyer and Marcu (2012)
- Wang and Manning (2012)
- Leusch and Ney (2009)
- Snover et al. (2009)