Class SamplingConfig

deepspeed/runtime/rollout/base.py:28–36

Sampling knobs that the trainer passes to ``generate`` each step.

@dataclass
class SamplingConfig:
    """Sampling knobs that the trainer passes to ``generate`` each step."""

    max_new_tokens: int
    temperature: float = 1.0
    top_p: float = 1.0
    top_k: int = -1
    n_samples_per_prompt: int = 1
    continuous_batch_size: Optional[int] = None


@dataclass