Functions — github.com/deepseek-ai/DualPipe

hub / github.com/deepseek-ai/DualPipe / functions

Functions70 in github.com/deepseek-ai/DualPipe

⨍ Functions 70 ◇ Types & classes 9

↓ 6 callersMethod_backward_chunk

(self, phase: int, enable_zb: bool = False, recv: bool = True, send: bool = True)

dualpipe/dualpipev.py:197

↓ 6 callersMethod_backward_chunk

(self, phase: int, enable_zb: bool = False, recv: bool = True, send: bool = True)

dualpipe/dualpipe.py:195

↓ 5 callersMethod_commit_and_wait_comm

(self)

dualpipe/dualpipev.py:279

↓ 5 callersMethod_commit_and_wait_comm

(self)

dualpipe/dualpipe.py:285

↓ 5 callersMethod_forward_chunk

(self, phase: int, recv: bool = True, send: bool = True)

dualpipe/dualpipev.py:187

↓ 5 callersMethod_forward_chunk

(self, phase: int, recv: bool = True, send: bool = True)

dualpipe/dualpipe.py:185

↓ 5 callersMethod_recv_forward

(self, phase: int)

dualpipe/dualpipev.py:233

↓ 5 callersMethod_recv_forward

(self, phase: int)

dualpipe/dualpipe.py:231

↓ 5 callersMethod_send_forward

(self, phase: int)

dualpipe/dualpipev.py:241

↓ 5 callersMethod_send_forward

(self, phase: int)

dualpipe/dualpipe.py:241

↓ 4 callersMethod_forward_backward_chunk

(self, phase0: int, phase1: int, recv0: bool = True)

dualpipe/dualpipev.py:207

↓ 4 callersMethod_forward_backward_chunk

(self, phase0: int, phase1: int, recv0: bool = True)

dualpipe/dualpipe.py:205

↓ 4 callersMethodbackward

(ctx, grad_output)

examples/example_dualpipe.py:21

↓ 4 callersFunctionrun_backward

(tensors: List[torch.Tensor], grad_tensors: List[torch.Tensor])

dualpipe/utils.py:36

↓ 4 callersFunctionscatter

(inputs, chunks, dim)

dualpipe/utils.py:62

↓ 3 callersMethod_forward_compute_chunk

(self, phase: int)

dualpipe/dualpipev.py:63

↓ 3 callersMethod_forward_compute_chunk

(self, phase: int)

dualpipe/dualpipe.py:67

↓ 3 callersMethod_send_backward

(self, phase: int)

dualpipe/dualpipev.py:265

↓ 3 callersMethod_send_backward

(self, phase: int)

dualpipe/dualpipe.py:269

↓ 3 callersMethod_weight_chunk

(self)

dualpipe/dualpipev.py:218

↓ 3 callersMethod_weight_chunk

(self)

dualpipe/dualpipe.py:216

↓ 3 callersMethodput

(cls, func: Callable)

dualpipe/utils.py:15

↓ 2 callersMethod_backward_compute_chunk

(self, phase: int, enable_zb: bool = False)

dualpipe/dualpipev.py:84

↓ 2 callersMethod_backward_compute_chunk

(self, phase: int, enable_zb: bool = False)

dualpipe/dualpipe.py:87

↓ 2 callersMethod_recv_backward

(self, phase: int)

dualpipe/dualpipev.py:254

↓ 2 callersMethod_recv_backward

(self, phase: int)

dualpipe/dualpipe.py:256

↓ 2 callersMethod_reset_states

(self)

dualpipe/dualpipev.py:43

↓ 2 callersMethod_reset_states

(self)

dualpipe/dualpipe.py:47

↓ 2 callersMethodbackward

(ctx, grad_output)

examples/example_dualpipev.py:21

↓ 2 callersMethodclear

(cls)

dualpipe/utils.py:31

↓ 2 callersMethodflush

(cls)

dualpipe/utils.py:19

↓ 2 callersFunctiongather

(micro_outputs, dim)

dualpipe/utils.py:74

↓ 2 callersMethodoverlapped_forward_backward

You should implement custom forward-backward overlap strategy. The code below is just an example.

examples/example_dualpipe.py:55

↓ 2 callersMethodpop

(cls)

dualpipe/utils.py:24

↓ 2 callersFunctionset_p2p_tensor_dtype

(dtype: torch.dtype)

dualpipe/comm.py:16

↓ 2 callersFunctionset_p2p_tensor_shapes

(shapes: List[Tuple[int]])

dualpipe/comm.py:11

↓ 2 callersMethodstep

Execute a training or inference step. Arguments: *inputs: Module inputs. Required only on the first rank. nu

dualpipe/dualpipev.py:288

↓ 2 callersMethodstep

Execute a training or inference step. Arguments: *inputs: Module inputs. Required only on the first/last ranks.

dualpipe/dualpipe.py:294

↓ 1 callersMethod_forward_backward_compute_chunk

(self, phase0: int, phase1: int)

dualpipe/dualpipev.py:120

↓ 1 callersMethod_forward_backward_compute_chunk

(self, phase0: int, phase1: int)

dualpipe/dualpipe.py:121

↓ 1 callersMethod_free_tensors

(self)

dualpipe/dualpipev.py:227

↓ 1 callersMethod_free_tensors

(self)

dualpipe/dualpipe.py:225

↓ 1 callersFunctionbuild_from_tensor_shapes

()

dualpipe/comm.py:21

↓ 1 callersFunctioncal_diff

(x: torch.Tensor, y: torch.Tensor)

examples/example_dualpipev.py:103

↓ 1 callersFunctioncal_diff

(x: torch.Tensor, y: torch.Tensor)

examples/example_dualpipe.py:103

↓ 1 callersFunctioncat_tensor

(x, dim)

dualpipe/utils.py:52

↓ 1 callersFunctionchunk_tensor

(x, chunks, dim)

dualpipe/utils.py:46

↓ 1 callersFunctioncriterion

(output: torch.Tensor, target: torch.Tensor)

examples/example_dualpipev.py:86

↓ 1 callersFunctioncriterion

(output: torch.Tensor, target: torch.Tensor)

examples/example_dualpipe.py:86

↓ 1 callersFunctionref_step

(x, l, model, chunks)

examples/example_dualpipev.py:90

↓ 1 callersFunctionref_step

(x, l, model, chunks)

examples/example_dualpipe.py:90

↓ 1 callersFunctiontest_dualpipe

(ngpus)

examples/example_dualpipe.py:195

↓ 1 callersFunctiontest_dualpipev

(ngpus)

examples/example_dualpipev.py:176

Method__init__

( self, modules: Tuple[nn.Module, nn.Module], batch_dim: int = 0, process_grou

dualpipe/dualpipev.py:12

Method__init__

( self, modules: Tuple[nn.Module, nn.Module], batch_dim: int = 0, process_grou

dualpipe/dualpipe.py:12

Method__init__

(self, hidden_size: int)

examples/example_dualpipev.py:43

Method__init__

(self, hidden_size: int)

examples/example_dualpipe.py:43

Functionappend_irecv

(ops: List[dist.P2POp], src: int, group: dist.ProcessGroup)

dualpipe/comm.py:25

Functionappend_isend

(ops: List[dist.P2POp], tensors: List[torch.Tensor], dst: int, group: dist.ProcessGroup)

dualpipe/comm.py:34

Methodforward

(ctx, input, weight)

examples/example_dualpipev.py:15

Methodforward

(self, input: torch.Tensor)

examples/example_dualpipev.py:38

Methodforward

(self, x: torch.Tensor)

examples/example_dualpipev.py:48

Methodforward

(ctx, input, weight)

examples/example_dualpipe.py:15

Methodforward

(self, input: torch.Tensor)

examples/example_dualpipe.py:38

Methodforward

(self, x: torch.Tensor)

examples/example_dualpipe.py:48

Methodgrad_weight_fn

()

examples/example_dualpipev.py:26

Methodgrad_weight_fn

()

examples/example_dualpipe.py:26

Functionmain

(rank, pp_size)

examples/example_dualpipev.py:109

Functionmain

(rank, pp_size)

examples/example_dualpipe.py:109

Methodoverlapped_forward_backward

You should implement custom forward-backward overlap strategy. The code below is just an example.

examples/example_dualpipev.py:55