Class ExportArguments

swift/arguments/export_args.py:16–150

ExportArguments is a dataclass that inherits from BaseArguments and MergeArguments. Args: output_dir (Optional[str]): Directory to save the exported results. Defaults to None, which automatically sets a path with an appropriate suffix. quant_method (Optional[str]): T

@dataclass
class ExportArguments(MergeArguments, BaseArguments):
    """ExportArguments is a dataclass that inherits from BaseArguments and MergeArguments.

    Args:
        output_dir (Optional[str]): Directory to save the exported results. Defaults to None, which automatically sets
            a path with an appropriate suffix.
        quant_method (Optional[str]): The quantization method. Can be 'awq', 'gptq', 'bnb', 'fp8', or 'gptq_v2'.
            Defaults to None. See examples for more details.
        quant_n_samples (int): Number of samples for GPTQ/AWQ calibration. Defaults to 256.
        quant_batch_size (int): The batch size for quantization. Defaults to 1.
        group_size (int): The group size for quantization. Defaults to 128.
        to_cached_dataset (bool): Whether to tokenize and export the dataset in advance as a cached dataset. Defaults
            to False. Note: You can specify the validation set content through
            `--split_dataset_ratio` or `--val_dataset`.
        to_ollama (bool): Whether to generate the `Modelfile` required by Ollama. Defaults to False.
        to_mcore (bool): Whether to convert Hugging Face format weights to Megatron-Core format. Defaults to False.
        to_hf (bool): Whether to convert Megatron-Core format weights to Hugging Face format. Defaults to False.
        mcore_model (Optional[str]): The path to the Megatron-Core format model. Defaults to None.
        mcore_adapter (Optional[str]): A list of adapter paths for the Megatron-Core format model. Defaults to [].
        thread_count (Optional[int]): The number of model shards when `to_mcore` is True. Defaults to None, which
            automatically sets the number based on the model size to keep the largest shard under 10GB.
        test_convert_precision (bool): Whether to test the precision error of weight conversion between Hugging Face
            and Megatron-Core formats. Defaults to False.
        test_convert_dtype (str): The dtype to use for the conversion precision test. Defaults to 'float32'.
        push_to_hub (bool): Whether to push the output to the Model Hub. Defaults to False. See examples for more
            details.
        hub_model_id (Optional[str]): The model ID for pushing to the Hub (e.g., 'user_name/repo_name' or 'repo_name').
            Defaults to None.
        hub_private_repo (bool): Whether the Hub repository is private. Defaults to False.
        commit_message (str): The commit message for pushing to the Hub. Defaults to 'update files'.
        to_peft_format (bool): Whether to export in PEFT format. This argument is for compatibility and currently has
            no effect. Defaults to False.
        exist_ok (bool): If the output_dir exists, do not raise an exception and overwrite its contents. Defaults to
            False.
    """
    output_dir: Optional[str] = None

    # awq/gptq
    quant_method: Literal['awq', 'gptq', 'bnb', 'fp8', 'gptq_v2'] = None
    quant_n_samples: int = 256
    quant_batch_size: int = 1
    group_size: int = 128

    # cached_dataset
    to_cached_dataset: bool = False
    template_mode: Literal['train', 'rlhf', 'kto'] = 'train'

    # ollama
    to_ollama: bool = False

    # megatron
    to_mcore: bool = False
    to_hf: bool = False
    mcore_model: Optional[str] = None
    mcore_adapter: Optional[str] = None
    thread_count: Optional[int] = None
    test_convert_precision: bool = False
    test_convert_dtype: str = 'float32'