MCPcopy
hub / github.com/mudler/LocalAI / GetGPUMemoryUsage

Function GetGPUMemoryUsage

pkg/xsysinfo/gpu.go:290–343  ·  view source on GitHub ↗

GetGPUMemoryUsage returns real-time GPU memory usage for all detected GPUs. It tries multiple vendor-specific tools in order: NVIDIA, AMD, Intel, Vulkan. Returns an empty slice if no GPU monitoring tools are available.

()

Source from the content-addressed store, hash-verified

288// It tries multiple vendor-specific tools in order: NVIDIA, AMD, Intel, Vulkan.
289// Returns an empty slice if no GPU monitoring tools are available.
290func GetGPUMemoryUsage() []GPUMemoryInfo {
291 var gpus []GPUMemoryInfo
292
293 // Try NVIDIA first
294 nvidiaGPUs := getNVIDIAGPUMemory()
295 if len(nvidiaGPUs) > 0 {
296 gpus = append(gpus, nvidiaGPUs...)
297 }
298
299 // XXX: Note - I could not test this with AMD and Intel GPUs, so I'm not sure if it works and it was added with the help of AI.
300
301 // Try AMD ROCm
302 amdGPUs := getAMDGPUMemory()
303 if len(amdGPUs) > 0 {
304 // Adjust indices to continue from NVIDIA GPUs
305 startIdx := len(gpus)
306 for i := range amdGPUs {
307 amdGPUs[i].Index = startIdx + i
308 }
309 gpus = append(gpus, amdGPUs...)
310 }
311
312 // Try Intel
313 intelGPUs := getIntelGPUMemory()
314 if len(intelGPUs) > 0 {
315 startIdx := len(gpus)
316 for i := range intelGPUs {
317 intelGPUs[i].Index = startIdx + i
318 }
319 gpus = append(gpus, intelGPUs...)
320 }
321
322 // Try NVIDIA integrated GPUs (Tegra Jetson, DGX Spark, Thor — unified memory).
323 // These either lack nvidia-smi or have it behave unreliably, so we detect
324 // them via SoC sysfs and report system RAM figures.
325 if len(gpus) == 0 {
326 integratedGPUs := getNVIDIAIntegratedGPUMemory()
327 gpus = append(gpus, integratedGPUs...)
328 }
329
330 // Try Vulkan as fallback for device detection (limited real-time data)
331 if len(gpus) == 0 {
332 vulkanGPUs := getVulkanGPUMemory()
333 gpus = append(gpus, vulkanGPUs...)
334 }
335
336 // Try Apple Silicon (macOS only)
337 if len(gpus) == 0 {
338 appleGPUs := getAppleGPUMemory()
339 gpus = append(gpus, appleGPUs...)
340 }
341
342 return gpus
343}
344
345// GetGPUAggregateInfo returns aggregate GPU information across all GPUs
346func GetGPUAggregateInfo() GPUAggregateInfo {

Callers 4

TotalAvailableVRAMFunction · 0.85
MinPerGPUVRAMFunction · 0.85
GetGPUAggregateInfoFunction · 0.85
GetResourceInfoFunction · 0.85

Calls 6

getNVIDIAGPUMemoryFunction · 0.85
getAMDGPUMemoryFunction · 0.85
getIntelGPUMemoryFunction · 0.85
getVulkanGPUMemoryFunction · 0.85
getAppleGPUMemoryFunction · 0.85

Tested by

no test coverage detected