Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/boyiwei/alignment-attribution-code
/ TokenizerWrapper
Class
TokenizerWrapper
lib/data.py:16–18 ·
view source on GitHub ↗
Source
from the content-addressed store, hash-verified
14
15
# Wrapper for tokenized input IDs
16
class
TokenizerWrapper:
17
def
__init__(self, input_ids):
18
self.input_ids = input_ids
19
20
21
# Load and process aligned dataset
Callers
nothing calls this directly
Calls
no outgoing calls
Tested by
no test coverage detected