MCPcopy Create free account
hub / github.com/THUDM/LongWriter / process_data

Method process_data

train/dataset.py:13–16  ·  view source on GitHub ↗
(self, filepath)

Source from the content-addressed store, hash-verified

11 self.labels = self.labels
12
13 def process_data(self, filepath):
14 input_ids = torch.from_numpy(np.load(os.path.join(filepath, 'inputs.npy')))
15 labels = torch.from_numpy(np.load(os.path.join(filepath, 'labels.npy')))
16 return input_ids, labels
17
18 def __getitem__(self, idx):
19 return {

Callers 1

__init__Method · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected