Translation Error Rate is an error metric for machine translation that measures the number of edits required to change a system output into one of the references Source: http://www.cs.umd.edu/~snover/tercom/ Paper: http://mt-archive.info/AMTA-2006-Snover.pdf Lower is better
(items)
| 102 | |
| 103 | @register_aggregation("ter") |
| 104 | def ter(items): |
| 105 | """Translation Error Rate is an error metric for machine translation that |
| 106 | measures the number of edits required to change a system output into one |
| 107 | of the references |
| 108 | Source: http://www.cs.umd.edu/~snover/tercom/ |
| 109 | Paper: http://mt-archive.info/AMTA-2006-Snover.pdf |
| 110 | |
| 111 | Lower is better |
| 112 | """ |
| 113 | refs = list(zip(*items))[0] |
| 114 | preds = list(zip(*items))[1] |
| 115 | refs, preds = _sacreformat(refs, preds) |
| 116 | return sacrebleu.corpus_ter(preds, refs).score |
| 117 | |
| 118 | |
| 119 | @register_metric( |
nothing calls this directly
no test coverage detected