MCPcopy Create free account
hub / github.com/google-research/language / encode_example

Method encode_example

language/serene/preprocessing.py:328–345  ·  view source on GitHub ↗
(self, *, claim, evidence, claim_label, evidence_label,
                     metadata, wikipedia_url, sentence_id)

Source from the content-addressed store, hash-verified

326 return cls(vocab_list=vocab_lines, tokenizer=tokenizer, **kwargs)
327
328 def encode_example(self, *, claim, evidence, claim_label, evidence_label,
329 metadata, wikipedia_url, sentence_id):
330 encoded_claim = self.encode(claim)
331 encoded_claim = encoded_claim[:self._max_claim_tokens]
332 evidence = create_evidence_input(
333 wikipedia_url if self._include_title else None,
334 sentence_id if self._include_sentence_id else None,
335 evidence
336 )
337 encoded_evidence = self.encode(evidence)
338 encoded_evidence = encoded_evidence[:self._max_evidence_tokens]
339 return {
340 'claim_text': encoded_claim,
341 'evidence_text': encoded_evidence,
342 'claim_label': claim_label,
343 'evidence_label': evidence_label,
344 'metadata': metadata,
345 }
346
347 def build_encoder_fn(self):
348

Callers

nothing calls this directly

Calls 2

encodeMethod · 0.95
create_evidence_inputFunction · 0.85

Tested by

no test coverage detected