Uses RequestFactory to construct Requests and returns an iterable of Requests which will be sent to the LM. :param doc: The document as returned from training_docs, validation_docs, or test_docs. :param ctx: str The context string, generat
(self, doc, ctx)
| 92 | return prompt.replace("[MASK]", self.doc_to_target(doc)[1:]) |
| 93 | |
| 94 | def construct_requests(self, doc, ctx): |
| 95 | """Uses RequestFactory to construct Requests and returns an iterable of |
| 96 | Requests which will be sent to the LM. |
| 97 | |
| 98 | :param doc: |
| 99 | The document as returned from training_docs, validation_docs, or |
| 100 | test_docs. |
| 101 | :param ctx: str |
| 102 | The context string, generated by fewshot_context. This includes the natural |
| 103 | language description, as well as the few shot examples, and the question |
| 104 | part of the document for `doc`. |
| 105 | """ |
| 106 | |
| 107 | ll_yes = rf.loglikelihood_rolling(ctx.replace("[MASK]", self.YES)) |
| 108 | ll_no = rf.loglikelihood_rolling(ctx.replace("[MASK]", self.NO)) |
| 109 | |
| 110 | return ll_yes, ll_no |
| 111 | |
| 112 | def process_results(self, doc, results): |
| 113 | """Take a single document and the LM results and evaluates, returning a |
nothing calls this directly
no test coverage detected