MCPcopy Create free account
hub / github.com/google-research/language / evaluate_predictions_impl

Function evaluate_predictions_impl

language/orqa/utils/eval_utils.py:77–94  ·  view source on GitHub ↗

Calculates and returns metrics.

(references,
                              predictions,
                              is_regex)

Source from the content-addressed store, hash-verified

75
76
77def evaluate_predictions_impl(references,
78 predictions,
79 is_regex):
80 """Calculates and returns metrics."""
81 missing_predictions = 0
82 correct = 0
83 for q, a in references.items():
84 if q in predictions:
85 correct += int(
86 is_correct(answers=a, prediction=predictions[q], is_regex=is_regex))
87 else:
88 missing_predictions += 1
89
90 return dict( # pytype: disable=bad-return-type # dict-kwargs
91 missing_predictions=missing_predictions,
92 num_correct=correct,
93 num_total=len(references),
94 accuracy=correct / float(len(references)))
95
96
97def evaluate_predictions(

Callers 1

evaluate_predictionsFunction · 0.85

Calls 1

is_correctFunction · 0.85

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…