MCPcopy Create free account
hub / github.com/bytedance/ABQ-LLM / get_gpu_memory

Function get_gpu_memory

algorithm/parallel_utils.py:56–70  ·  view source on GitHub ↗
()

Source from the content-addressed store, hash-verified

54
55
56def get_gpu_memory():
57 memory_info = []
58 gpu_memory_info = nvidia_smi_memory_info()
59
60 try:
61 gpu_index = [int(k) for k in os.environ['CUDA_VISIBLE_DEVICES'].split(',')]
62 except KeyError:
63 gpu_index = [x["id"] for x in gpu_memory_info]
64
65 for gpu_id, i in enumerate( gpu_index):
66 gpu = gpu_memory_info[i]
67 total_memory = gpu["total_memory"]
68 used_memory = gpu["used_memory"]
69 memory_info.append((gpu_id, total_memory, used_memory))
70 return memory_info
71
72
73def get_lowest_occupied_gpu(wait_memory=1000):

Callers 4

get_lowest_occupied_gpuFunction · 0.85
get_all_gpu_free_memoryFunction · 0.85
assign_layers_to_gpusFunction · 0.85
parallel_utils.pyFile · 0.85

Calls 2

nvidia_smi_memory_infoFunction · 0.85
appendMethod · 0.45

Tested by

no test coverage detected