MCPcopy Create free account
hub / github.com/boyiwei/alignment-attribution-code / TokenizerWrapper

Class TokenizerWrapper

lib/data.py:16–18  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

14
15# Wrapper for tokenized input IDs
16class TokenizerWrapper:
17 def __init__(self, input_ids):
18 self.input_ids = input_ids
19
20
21# Load and process aligned dataset

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected