Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of the metric for that one document :param doc: The document as returned from training_docs, validation_docs, or test_docs.
(self, doc, results)
| 85 | return ll_no, ll_yes |
| 86 | |
| 87 | def process_results(self, doc, results): |
| 88 | """Take a single document and the LM results and evaluates, returning a |
| 89 | dict where keys are the names of submetrics and values are the values of |
| 90 | the metric for that one document |
| 91 | |
| 92 | :param doc: |
| 93 | The document as returned from training_docs, validation_docs, or test_docs. |
| 94 | :param results: |
| 95 | The results of the requests created in construct_requests. |
| 96 | """ |
| 97 | ll_no, ll_yes = results |
| 98 | gold = doc["label"] |
| 99 | pred = int(ll_yes > ll_no) |
| 100 | question_id = self._question2id(doc) |
| 101 | items = (gold, pred, question_id) |
| 102 | return {"em": items, "f1": items} |
| 103 | |
| 104 | def _question2id(self, doc): |
| 105 | """Returns an identifier for the question in the given document.""" |
nothing calls this directly
no test coverage detected