MCPcopy Create free account
hub / github.com/huggingface/transformers / get_dataset

Method get_dataset

examples/seq2seq/finetune.py:180–190  ·  view source on GitHub ↗
(self, type_path)

Source from the content-addressed store, hash-verified

178 return self.validation_epoch_end(outputs, prefix="test")
179
180 def get_dataset(self, type_path) -> SummarizationDataset:
181 n_obs = self.n_obs[type_path]
182 max_target_length = self.target_lens[type_path]
183 dataset = SummarizationDataset(
184 self.tokenizer,
185 type_path=type_path,
186 n_obs=n_obs,
187 max_target_length=max_target_length,
188 **self.dataset_kwargs,
189 )
190 return dataset
191
192 def get_dataloader(self, type_path: str, batch_size: int, shuffle: bool = False) -> DataLoader:
193 dataset = self.get_dataset(type_path)

Callers 1

get_dataloaderMethod · 0.95

Calls 1

Tested by

no test coverage detected