Function dask_client_create

pyxrf/core/map_processing.py:23–71

Create Dask client object. The function is trivial and introduced so that Dask client is created in uniform way throughout the program. The client is configured to keep temporary data in `~/.dask` directory instead of the current directory. Creating new Dask client may be costl

(**kwargs)

def dask_client_create(**kwargs):
    """
    Create Dask client object. The function is trivial and introduced so that
    Dask client is created in uniform way throughout the program.
    The client is configured to keep temporary data in `~/.dask` directory
    instead of the current directory.

    Creating new Dask client may be costly (a few extra seconds). If computations require
    multiple calls to functions that are running on Dask, overhead can be reduced by creating
    one common Dask client and supplying the reference to the client as a parameter in each
    function call.

    .. code:: python

        client = dask_client_create()  # Create Dask client
        # <-- code that runs computations -->
        client.close()  # Close Dask client

    Parameters
    ----------
    kwargs: dict, optional
        kwargs will be passed to the Dask client constructor. No extra parameters are needed
        in most cases.

    Returns
    -------
    client: dask.distributed.Client
        Dask client object
    """
    _kwargs = {"processes": True, "silence_logs": logging.ERROR}
    _kwargs.update(kwargs)

    logger.info("Creating Dask Client ...")

    dask.config.set(shuffle="disk")

    current_os = platform.system()
    if current_os == "Linux":
        tmp_dir = tempfile.gettempdir()
        user_name = getpass.getuser()
        path_dask_data = os.path.join(tmp_dir, user_name, "dask")
        os.makedirs(path_dask_data, exist_ok=True)
    else:
        path_dask_data = os.path.expanduser("~/.dask")

    dask.config.set({"temporary_directory": path_dask_data})

    client = Client(**_kwargs)
    return client


class TerminalProgressBar: