Function spectrogram_batch

src/transformers/src/transformers/audio_utils.py:584–788

Calculates spectrograms for a list of waveforms using the Short-Time Fourier Transform, optimized for batch processing. This function extends the capabilities of the `spectrogram` function to handle multiple waveforms efficiently by leveraging broadcasting. It supports generating vario

(
    waveform_list: List[np.ndarray],
    window: np.ndarray,
    frame_length: int,
    hop_length: int,
    fft_length: Optional[int] = None,
    power: Optional[float] = 1.0,
    center: bool = True,
    pad_mode: str = "reflect",
    onesided: bool = True,
    preemphasis: Optional[float] = None,
    mel_filters: Optional[np.ndarray] = None,
    mel_floor: float = 1e-10,
    log_mel: Optional[str] = None,
    reference: float = 1.0,
    min_value: float = 1e-10,
    db_range: Optional[float] = None,
    remove_dc_offset: Optional[bool] = None,
    dtype: np.dtype = np.float32,
)

def spectrogram_batch(
    waveform_list: List[np.ndarray],
    window: np.ndarray,
    frame_length: int,
    hop_length: int,
    fft_length: Optional[int] = None,
    power: Optional[float] = 1.0,
    center: bool = True,
    pad_mode: str = "reflect",
    onesided: bool = True,
    preemphasis: Optional[float] = None,
    mel_filters: Optional[np.ndarray] = None,
    mel_floor: float = 1e-10,
    log_mel: Optional[str] = None,
    reference: float = 1.0,
    min_value: float = 1e-10,
    db_range: Optional[float] = None,
    remove_dc_offset: Optional[bool] = None,
    dtype: np.dtype = np.float32,
) -> List[np.ndarray]:
    """
    Calculates spectrograms for a list of waveforms using the Short-Time Fourier Transform, optimized for batch processing.
    This function extends the capabilities of the `spectrogram` function to handle multiple waveforms efficiently by leveraging broadcasting.

    It supports generating various types of spectrograms:

        - amplitude spectrogram (`power = 1.0`)
        - power spectrogram (`power = 2.0`)
        - complex-valued spectrogram (`power = None`)
        - log spectrogram (use `log_mel` argument)
        - mel spectrogram (provide `mel_filters`)
        - log-mel spectrogram (provide `mel_filters` and `log_mel`)

    How this works:

        1. The input waveform is split into frames of size `frame_length` that are partially overlapping by `frame_length
            - hop_length` samples.
        2. Each frame is multiplied by the window and placed into a buffer of size `fft_length`.
        3. The DFT is taken of each windowed frame.
        4. The results are stacked into a spectrogram.

    We make a distinction between the following "blocks" of sample data, each of which may have a different lengths:

      - The analysis frame. This is the size of the time slices that the input waveform is split into.
      - The window. Each analysis frame is multiplied by the window to avoid spectral leakage.
      - The FFT input buffer. The length of this determines how many frequency bins are in the spectrogram.

    In this implementation, the window is assumed to be zero-padded to have the same size as the analysis frame. A
    padded window can be obtained from `window_function()`. The FFT input buffer may be larger than the analysis frame,
    typically the next power of two.

    Note: This function is designed for efficient batch processing of multiple waveforms but retains compatibility with individual waveform processing methods like `librosa.stft`.

    Args:
        waveform_list (`List[np.ndarray]` with arrays of shape `(length,)`):
            The list of input waveforms, each a single-channel (mono) signal.
        window (`np.ndarray` of shape `(frame_length,)`):
            The windowing function to apply, including zero-padding if necessary.