The Bilingual Evaluation Understudy Score, or BLEU for short, is a metric for evaluating a generated sentence to a reference sentence. It counts matching n-grams in the candidate translation to n-grams in the reference text, where 1-gram or unigram would be each token and a bigram compar
(items)
| 109 | |
| 110 | |
| 111 | def bleu(items): |
| 112 | """The Bilingual Evaluation Understudy Score, or BLEU for short, is a metric |
| 113 | for evaluating a generated sentence to a reference sentence. It counts matching |
| 114 | n-grams in the candidate translation to n-grams in the reference text, where |
| 115 | 1-gram or unigram would be each token and a bigram comparison would be each |
| 116 | word pair. The comparison is made regardless of word order |
| 117 | Source: https://machinelearningmastery.com/calculate-bleu-score-for-text-python/ |
| 118 | Paper: https://www.aclweb.org/anthology/P02-1040/ |
| 119 | |
| 120 | Higher is better |
| 121 | """ |
| 122 | refs = list(zip(*items))[0] |
| 123 | preds = list(zip(*items))[1] |
| 124 | refs, preds = _sacreformat(refs, preds) |
| 125 | return sacrebleu.corpus_bleu(preds, refs).score |
| 126 | |
| 127 | |
| 128 | def chrf(items): |
nothing calls this directly
no test coverage detected