MCPcopy Create free account
hub / github.com/deepspeedai/DeepSpeed / graph_process

Function graph_process

deepspeed/runtime/utils.py:92–107  ·  view source on GitHub ↗
(replay_first_step, func, *args, **kwargs)

Source from the content-addressed store, hash-verified

90
91
92def graph_process(replay_first_step, func, *args, **kwargs):
93 # `func` should only contain operations on the GPU
94 # Please ensure that the memory address of the data required by 'func' remains constant
95 if func.__name__ not in graph_cache:
96 cuda_stream = get_accelerator().Stream()
97 cuda_stream.wait_stream(get_accelerator().current_stream())
98 with get_accelerator().stream(cuda_stream):
99 func(*args, **kwargs)
100 get_accelerator().current_stream().wait_stream(cuda_stream)
101 graph_cache[func.__name__] = get_accelerator().create_graph()
102 with get_accelerator().capture_to_graph(graph_cache[func.__name__]):
103 func(*args, **kwargs)
104 if replay_first_step:
105 get_accelerator().replay_graph(graph_cache[func.__name__])
106 else:
107 get_accelerator().replay_graph(graph_cache[func.__name__])
108
109
110def noop_decorator(func):

Callers 3

update_hp_gradsMethod · 0.90

Calls 9

get_acceleratorFunction · 0.90
funcFunction · 0.85
wait_streamMethod · 0.80
StreamMethod · 0.45
current_streamMethod · 0.45
streamMethod · 0.45
create_graphMethod · 0.45
capture_to_graphMethod · 0.45
replay_graphMethod · 0.45

Tested by

no test coverage detected