Simple gguf LLM Qwen3-VL, Qwen3.5, Qwen3.6, Gemma4 and others model loader for Comfy-UI.

Why need this version?

This version was created to meet my requirements: 1. The model must support gguf (gguf models run faster than transformer models). 2. The model must support the Qwen3-VL, Qwen3.5, Qwen3.6, Gemma4 multimodal model. 3. The node should be easily adaptable to work with any new released model. 4. After running, the node must be completely cleared from memory, leaving no garbage behind. This is important. Next come very resource-intensive processes that require ALL the memory. (Yes, the model will have to be reloaded every time, but this is better than storing the model as dead weight while heavier tasks suffer from lack of memory and run slower). In the latest update added a new keep_vram mode, which allows you to keep the model from being unloaded from memory. Convenient for small models and batch modes. 5. No auto-loaded models. You can use any models you already have (from LM Studio etc). Just simply specify their path on the disk to config.

Last update:

Nightly (tests)

• Add dynamic image, audio, video input, Add "user_prompt_template" input, Add "bypass" input
• New design for LLM Config
• Added new configurator 🌐 LLM Config and 🌐 LLM Prompt Preset

The new advanced configurator brings a completely redesigned configuration experience with the following advantages:

Key Features:

1. Built-in Preset Management - Direct access to JSON preset files from within ComfyUI. You can now add new models, delete, or rename presets without manually editing JSON files. The preset browser integrates seamlessly with save/rename/delete buttons right in the node interface.
2. Complete Parameter Access - All 73+ parameters currently supported by the system are available in one place. To prevent overwhelming users, parameters are organized into 9 collapsible groups (Model & Paths, Memory & Context, Sampling & Generation, etc.). Only essential parameters are visible by default; advanced settings remain hidden until needed.
3. Windows File Browser - For Windows users, dedicated Browse Model and Browse MMProj buttons allow selecting GGUF files from anywhere on your disk, not just ComfyUI's predefined folders. No more manual path typing!
4. Flexible Widget Layout - The configurator now supports easy rearrangement and insertion of new (future) parameter. Previously, adding a new parameter to the middle would displace all the others, corrupting old saves. With the new approach, this problem no longer exists.

💡 TIP: If you need to move preset lists to the top level of the subgraph, use widgets of the LLM Inference node — they work in the classic Comfi-UI way.

• Added words_to_ban config (logit_bias).
• Added 📸 Simple Gif Maker node.
• Added 📸 Load Video Fragment node. What makes this node different from others? It can download a file of any size, but it does NOT copy it to the input folder. At the same time it has browse button, crop, resize and timeline.
• Added Bernini presets.
• Add _user_prompt_template functionality. Now you can modify the user_prompt using a specified template (the name of which matches the system_preset)
• Add variables input. You can now set any user placeholders in {} in the system and user prompts.
• Added autocomplete placeholders width, height, image_num, ref_num, audio_num, frame_num, user_prompt. By default, placeholder replase is disabled for backward compatibility. It can be enabled by passing user variables to the variables input (just like the config input) or by using _user_prompt_template, or by forcing it by entering "enable_variables": true, in config.
• Added the add_image_id, add_audio_id, and add_frame_id configurations, which allow you to number the corresponding content according to a specified template before inserting it.
• An additional configuration file has been added to the following path: ComfyUI\user\SimpleQwenVL_configs\system_prompts_user.json.
• improvement of video input (part 1)
• fix UnicodeDecodeError error in subprocess
• add node Ideogram 4 JSON Preview and Ideogram 4 JSON Swap XY Coordinates
• present_penalty/presence_penalty issue

16.06.2026 - V3.9 - Fix f-string: unmatched caused by nested double quotes - Fix disappearance of "\n" line breaks in raw_mode

22.05.2026 - V3.8 - Added example qwen_vl_test_translate - Added modes: save1, save2, save3 - Added example qwen_vl_test_image_storytaler - Added utils: Simple Text To Batch, Simple Text Insert, Simple Text Replace, Simple Join Strings - Added simple LLM configurator - Improved error output

03.05.2026 - V3.7 - Added force_mmproj settings. - Added support for n_cpu_moe, cpu_moe. Requires llama_cpp_python update to 0.3.37+. See the limitations in the Speed ​​test and memory overflow problem section below. - Standard parameter names are now supported - Added debug calculate token/sec - Added options for running encoder (to obtain embeddings or conditioning) - Added video input (while llama.cpp doesn't have native support yet, you can pass a reduced set of frames, see example) - Added audio input (see example) - Added split_mode settings for multi GPU

04.04.2026 - V3.6 - Add Gemma4 support. - Fix raw_mode in text mode.

08.03.2026 - V3.5 - TurboQuants feature (for now requires a fork of llama.cpp) - Adding a new mode "raw_mode": true which allows you to set custom prompt templates. The Joycaption model now works correctly (see new configs below). - Three execution modes have been added: subprocess — inference runs in a separate process (safe, isolated); direct_clean — in the main process with model unloading after each run; keep_vram — the model remains in VRAM for repeated use. - Added config_override - the ability to add/override any configuration parameters via a text input directly in the node - Integrated json_repair to automatically repair invalid JSON in config_override and system_prompts_user.json - Expanded documentation on configuration fields and operating modes

04.03.2026 - V3.2 - Added support for Qwen3.5

Correct installation of llama-cpp-python:

Qwen3 support hasn't been added to the standard library, llama-cpp-python, which is downloaded via pip install llama-cpp-python - this didn't work. The standard version llama-cpp-python hasn't been updated for a long time. llama-cpp-python 0.3.16 last commit on Aug 15, 2025 and it doesn't support qwen3.

Check the version number of llama-cpp-python from JamePeng you're using: - Version 0.3.17 or latest supports qwen3-VL. - Version 0.3.30 or latest supports qwen3.5. - Version 0.3.35 or latest supports gemma4.

Variant 1 - Download WHL

Download WHL packages for your configuration

• https://github.com/JamePeng/llama-cpp-python/releases

For example:

cd *path_to_comfyui*\python_embeded

python -m pip install json_repair,colorama

python -m pip install temp\llama_cpp_python-0.3.18-cp313-cp313-win_amd64.whl

💡 WARNING: These ready-made basic VHLs may not have CPU acceleration implementations. Therefore, installing them may not yield any benefit from n_cpu_moe or cpu_moe. Use VHL with optimizations enabled, or better yet, compile the project yourself for your hardware.

💡 Tip: In subprocess mode, you can launch it immediately. In other modes, you need to restart Comfy-UI.

Variant 2 - Build from source code (I recommend this variant)

Installing software before compilation

1. Check that you have CUDA Toolkit installed. For example: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0
2. Try installing: https://developer.nvidia.com/cuda-downloads
3. Check that the PATH in Environment Variables includes the CUDA Toolkit bin folder (For example: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0\bin).

4. After installing CUDA Toolkit, restart your computer.

5. Check that the NVIDIA Driver and CUDA Toolkit versions match (the driver can and most often should be newer than the CUDA Toolkit version): Run command in CMD nvidia-smi.

6. Check that you have Visual C++ Redistributable installed.

7. Try installing: https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170

8. Install both versions (x86 and x64).

9. Check that you have Visual Studio 2022 installed.

10. Install Visual Studio 2022.
11. Install the following packages (they will not be installed by default):

☑ Desktop development with C++ (in Workloads tab).

☑ MSVC v143 - VS 2022 C++ x64/x86 build tools (in Individual components tab).

☑ Windows 10/11 SDK (in Individual components tab).

☑ CMake tools for Visual Studio (in Individual components tab).

• The environment variable for MSVC is not added to the PATH by default. Run this command every time in your terminal before compiling: call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

• If you use python_embeded for Comfy-UI, may need to add missing libs folders: python_embeded\include, python_embeded\libs (Not Lib\site-packages), python_embeded\DLLs:

• From here https://github.com/astral-sh/python-build-standalone/releases download Python appropriate version (for example cpython-3.13.11+20251217-x86_64-pc-windows-msvc-install_only.tar.gz)
• unzip and copy the necessary folders to python_embeded.

Build llama-cpp-python from source code

1. Clone the repositories using Git:
2. https://github.com/JamePeng/llama-cpp-python
3. https://github.com/ggml-org/llama.cpp

git clone https://github.com/JamePeng/llama-cpp-python.git
git clone https://github.com/ggml-org/llama.cpp.git

1. Move the second project llama.cpp\ in the llama-cpp-python\vendor\ folder

2. Automatically set the paths to MSVC (Windows only):

call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

1. Optional: For fast build with Ninja

Using Ninja enables parallel compilation across CPU cores, significantly reducing build time (but may increase CPU temperature). Verify Ninja is installed with Visual Studio 2022:

ninja --version
1.12.1

• Configure environment variables (replace 32 with your desired number of cores):

set CMAKE_GENERATOR=Ninja
set MAX_JOBS=16

1. Go to the llama-cpp-python folder

cd *path_to_src*\llama-cpp-python

1. Set CUDA support and install the package:

*path_to_comfyui*\python -m pip install json_repair,colorama

set CMAKE_ARGS=-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_BUILD_TYPE=Release
*path_to_comfyui*\python_embeded\python -m pip install . 

✅ The command above is for embedded Python (typical for ComfyUI). Adjust the Python path if you're using a system or virtual environment.

Replace 120 with your сompute сapability number:

RTX 50-series (Blackwell) → 120
RTX 40-series → 89
RTX 30-series → 86
RTX 20-series → 75

⏱️ Build time: Without Ninja, compilation may take 30–60 minutes depending on your hardware.

⏱️ Build time: With Ninja, compilation may take 1–2 minutes depending on your hardware.

💡 Tip: In subprocess mode, you can launch it immediately. In other modes, you need to restart Comfu-ui.

Simple bat file for fast rebuild

cd llama-cpp-python\vendor\llama.cpp\
git pull
cd ..\..\
git pull --rebase
call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
set CMAKE_GENERATOR=Ninja
set MAX_JOBS=16
set CMAKE_ARGS=-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_BUILD_TYPE=Release
H:\ComfyUI128\python_embeded\python.exe -m pip install . --no-cache-dir --no-build-isolation
pause

✅ The command above is for embedded Python (typical for ComfyUI). Adjust the Python path if you're using a system or virtual environment.

Replace 120 with your сompute сapability number:

RTX 50-series (Blackwell) → 120
RTX 40-series → 89
RTX 30-series → 86
RTX 20-series → 75

💡 Tip: In subprocess mode, you can launch it immediately. In other modes, you need to restart Comfu-ui.

CUDA Support

This project requires CUDA runtime libraries. They can be sourced from: - The CUDA Toolkit: https://developer.nvidia.com/cuda-downloads (recommended) - OR an existing PyTorch installation

💡 Tip: If you use ComfyUI, you likely already have PyTorch. In that case, you probably don't need to install the CUDA Toolkit separately — the necessary libraries will be found automatically.

💡 Tip: After installing CUDA Toolkit, restart your computer.

Installation of ComfyUI_Simple_Qwen3-VL-gguf:

1.Installation to custom_nodes - Use ComfyUI Manager and find ComfyUI_Simple_Qwen3-VL-gguf - OR copy this project to the folder path_to_comfyui\ComfyUI\custom_nodes

  cd path_to_comfyui\ComfyUI\custom_nodes
  git clone https://github.com/KLL535/ComfyUI_Simple_Qwen3-VL-gguf

1. Restart ComfyUI. We check in the console that custom nodes are loading without errors.
2. Restarting the frontend (F5)

Implementation Features:

The node is split into two parts. All work is isolated in a subprocess. Why? To ensure everything is cleaned up and nothing unnecessary remains in memory after this node runs and llama.cpp. I've often encountered other nodes leaving something behind, and that's unaccepta