MCPcopy
hub / github.com/hpcaitech/ColossalAI / prepare_dataloader

Method prepare_dataloader

colossalai/booster/plugin/plugin_base.py:77–90  ·  view source on GitHub ↗

Prepare a dataloader for distributed training. The dataloader will be wrapped by `torch.utils.data.DataLoader`

(
        self,
        dataset: Dataset,
        batch_size: int,
        shuffle: bool = False,
        seed: int = 1024,
        drop_last: bool = False,
        pin_memory: bool = False,
        num_workers: int = 0,
        **kwargs,
    )

Source from the content-addressed store, hash-verified

75
76 @abstractmethod
77 def prepare_dataloader(
78 self,
79 dataset: Dataset,
80 batch_size: int,
81 shuffle: bool = False,
82 seed: int = 1024,
83 drop_last: bool = False,
84 pin_memory: bool = False,
85 num_workers: int = 0,
86 **kwargs,
87 ):
88 """Prepare a dataloader for distributed training. The dataloader will be wrapped by
89 `torch.utils.data.DataLoader`
90 """

Callers 15

trainFunction · 0.45
mainFunction · 0.45
mainFunction · 0.45
mainFunction · 0.45
train_dataloaderMethod · 0.45
val_dataloaderMethod · 0.45
test_dataloaderMethod · 0.45
check_torch_ddp_no_syncFunction · 0.45
train_dataloaderMethod · 0.45
val_dataloaderMethod · 0.45
test_dataloaderMethod · 0.45

Calls

no outgoing calls

Tested by 6

test_dataloaderMethod · 0.36
check_torch_ddp_no_syncFunction · 0.36
test_dataloaderMethod · 0.36
test_dataloaderMethod · 0.36
test_dataloaderMethod · 0.36