Function load_inline

python/tvm_ffi/cpp/extension.py:989–1146

Compile, build and load a C++/CUDA module from inline source code. This function compiles the given C++ and/or CUDA source code into a shared library. Both ``cpp_sources`` and ``cuda_sources`` are compiled to an object file, and then linked together into a shared library. It's possible to o

(  # noqa: PLR0913
    name: str,
    *,
    cpp_sources: Sequence[str] | str | None = None,
    cuda_sources: Sequence[str] | str | None = None,
    functions: Mapping[str, str] | Sequence[str] | str | None = None,
    extra_cflags: Sequence[str] | None = None,
    extra_cuda_cflags: Sequence[str] | None = None,
    extra_ldflags: Sequence[str] | None = None,
    extra_include_paths: Sequence[str] | None = None,
    build_directory: str | None = None,
    embed_cubin: Mapping[str, bytes] | None = None,
    keep_module_alive: bool = True,
    backend: str | None = None,
)

def load_inline(  # noqa: PLR0913
    name: str,
    *,
    cpp_sources: Sequence[str] | str | None = None,
    cuda_sources: Sequence[str] | str | None = None,
    functions: Mapping[str, str] | Sequence[str] | str | None = None,
    extra_cflags: Sequence[str] | None = None,
    extra_cuda_cflags: Sequence[str] | None = None,
    extra_ldflags: Sequence[str] | None = None,
    extra_include_paths: Sequence[str] | None = None,
    build_directory: str | None = None,
    embed_cubin: Mapping[str, bytes] | None = None,
    keep_module_alive: bool = True,
    backend: str | None = None,
) -> Module:
    """Compile, build and load a C++/CUDA module from inline source code.

    This function compiles the given C++ and/or CUDA source code into a shared library. Both ``cpp_sources`` and
    ``cuda_sources`` are compiled to an object file, and then linked together into a shared library. It's possible to only
    provide cpp_sources or cuda_sources.

    The ``functions`` parameter is used to specify which functions in the source code should be exported to the tvm ffi
    module. It can be a mapping, a sequence, or a single string. When a mapping is given, the keys are the names of the
    exported functions, and the values are docstrings for the functions. When a sequence of string is given, they are
    the function names needed to be exported, and the docstrings are set to empty strings. A single function name can
    also be given as a string, indicating that only one function is to be exported.

    Extra compiler and linker flags can be provided via the ``extra_cflags``, ``extra_cuda_cflags``, and ``extra_ldflags``
    parameters. The default flags are generally sufficient for most use cases, but you may need to provide additional
    flags for your specific use case.

    The include dir of tvm ffi and dlpack are used by default for the compiler to find the headers. Thus, you can
    include any header from tvm ffi in your source code. You can also provide additional include paths via the
    ``extra_include_paths`` parameter and include custom headers in your source code.

    The compiled shared library is cached in a cache directory to avoid recompilation. The `build_directory` parameter
    is provided to specify the build directory. If not specified, a default tvm ffi cache directory will be used.
    The default cache directory can be specified via the `TVM_FFI_CACHE_DIR` environment variable. If not specified,
    the default cache directory is ``~/.cache/tvm-ffi``.

    Parameters
    ----------
    name
        The name of the tvm ffi module.
    cpp_sources
        The C++ source code. It can be a list of sources or a single source.
    cuda_sources
        The CUDA source code. It can be a list of sources or a single source.
    functions
        The functions in cpp_sources or cuda_source that will be exported to the tvm ffi module. When a mapping is
        given, the keys are the names of the exported functions, and the values are docstrings for the functions
        (use an empty string to skip documentation for specific functions). When a sequence or a single string is given, they are
        the functions needed to be exported, and the docstrings are set to empty strings. A single function name can
        also be given as a string. When cpp_sources is given, the functions must be declared (not necessarily defined)
        in the cpp_sources. When cpp_sources is not given, the functions must be defined in the cuda_sources. If not
        specified, no function will be exported.
    extra_cflags
        The extra compiler flags for C++ compilation.