MCPcopy Create free account
hub / github.com/appdevforall/CodeOnTheGo / select_weight_buft

Function select_weight_buft

subprojects/llama.cpp/src/llama-model.cpp:313–324  ·  view source on GitHub ↗

find the first buffer type in the list that can use the tensor

Source from the content-addressed store, hash-verified

311
312// find the first buffer type in the list that can use the tensor
313static ggml_backend_buffer_type_t select_weight_buft(const llama_hparams & hparams, ggml_tensor * tensor, ggml_op op, const buft_list_t & buft_list) {
314 GGML_ASSERT(!buft_list.empty());
315 for (const auto & cur : buft_list) {
316 ggml_backend_dev_t cur_dev = cur.first;
317 ggml_backend_buffer_type_t cur_buft = cur.second;
318 if (weight_buft_supported(hparams, tensor, op, cur_buft, cur_dev)) {
319 return cur_buft;
320 }
321 }
322
323 return nullptr;
324}
325
326// CPU: ACCEL -> GPU host -> CPU extra -> CPU
327static buft_list_t make_cpu_buft_list(const std::vector<ggml_backend_dev_t> & devices, bool use_extra_bufts, bool no_host) {

Callers 1

load_tensorsMethod · 0.85

Calls 2

weight_buft_supportedFunction · 0.85
emptyMethod · 0.65

Tested by

no test coverage detected