MCPcopy Create free account
hub / github.com/deepspeedai/DeepSpeed / _graph_replay

Method _graph_replay

deepspeed/inference/engine.py:516–524  ·  view source on GitHub ↗
(self, *inputs, **kwargs)

Source from the content-addressed store, hash-verified

514 self.cuda_graph_created = True
515
516 def _graph_replay(self, *inputs, **kwargs):
517 for i in range(len(inputs)):
518 if torch.is_tensor(inputs[i]):
519 self.static_inputs[i].copy_(inputs[i])
520 for k in kwargs:
521 if torch.is_tensor(kwargs[k]):
522 self.static_kwargs[k].copy_(kwargs[k])
523 get_accelerator().replay_graph(self._cuda_graphs)
524 return self.static_output
525
526 def model_times(self):
527 assert self.model_profile_enabled, "model profiling is not enabled"

Callers 1

forwardMethod · 0.95

Calls 3

get_acceleratorFunction · 0.90
copy_Method · 0.80
replay_graphMethod · 0.45

Tested by

no test coverage detected