MCPcopy Create free account
hub / github.com/THUDM/LongWriter / process_data

Method process_data

train/dataset.py:33–36  ·  view source on GitHub ↗
(self, filepath)

Source from the content-addressed store, hash-verified

31 self.labels = self.labels
32
33 def process_data(self, filepath):
34 input_ids = torch.from_numpy(np.load(os.path.join(filepath, 'inputs_sort.npy')))
35 labels = torch.from_numpy(np.load(os.path.join(filepath, 'labels_sort.npy')))
36 return input_ids, labels
37
38 def __getitem__(self, idx):
39 return {

Callers 1

__init__Method · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected