Class ENABLE_CUDA_KERNEL

ppq/api/interface.py:915–935

Any code surrounded by with ENABLE_CUDA_KERNEL(): will invoke ppq's kernel functions for speed boost. This is a helper class for invoking highly-effcient custimized cuda kernel. PPQ developer team has implemented a series of quantization related cuda kernel, They are 5-100

class ENABLE_CUDA_KERNEL:
    """ Any code surrounded by 
    with ENABLE_CUDA_KERNEL():
    will invoke ppq's kernel functions for speed boost.
    
    This is a helper class for invoking highly-effcient custimized cuda
    kernel. PPQ developer team has implemented a series of quantization related
    cuda kernel, They are 5-100x faster than torch kernels, with less gpu
    memory cost.
    """
    def __init__(self) -> None:
        from ppq.core.ffi import CUDA_COMPLIER
        CUDA_COMPLIER.complie()
        self._state = False

    def __enter__(self):
        self._state = PPQ_CONFIG.USING_CUDA_KERNEL
        PPQ_CONFIG.USING_CUDA_KERNEL = True

    def __exit__(self, *args):
        PPQ_CONFIG.USING_CUDA_KERNEL = self._state


class DISABLE_CUDA_KERNEL: