Function init_process_group

Initializes the default process group. Args: local_rank (int): the rank on the current local machine. local_world_size (int): the world size (number of processes running) on the current local machine. shard_id (int): the shard index (machine rank) of the curr

(
    local_rank,
    local_world_size,
    shard_id,
    num_shards,
    init_method,
    dist_backend="nccl",
)