Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/VITA-MLLM/VITA-Audio
/ functions
Functions
566 in github.com/VITA-MLLM/VITA-Audio
⨍
Functions
566
◇
Types & classes
121
↳
Endpoints
4
↓ 1 callers
Function
preprocess
( sample, tokenizer: transformers.PreTrainedTokenizer, image_token_length: int, default_system
vita_audio/data/dataset_deepseek.py:353
↓ 1 callers
Function
preprocess
( sample, tokenizer: transformers.PreTrainedTokenizer, image_token_length: int, default_system
vita_audio/data/dataset_hunyuan.py:328
↓ 1 callers
Function
preprocess
( sample, tokenizer: transformers.PreTrainedTokenizer, image_token_length: int, default_system
vita_audio/data/dataset_cosyvoice2.py:330
↓ 1 callers
Function
preprocess
( sample, tokenizer: transformers.PreTrainedTokenizer, image_token_length: int, default_system
vita_audio/data/dataset_llama3.py:327
↓ 1 callers
Function
preprocess
( source, tokenizer: transformers.PreTrainedTokenizer, image_token_length: int, system_message
vita_audio/data/dataset_mistral.py:261
↓ 1 callers
Function
preprocess
( sample, tokenizer: transformers.PreTrainedTokenizer, image_token_length: int, default_system
vita_audio/data/dataset_qwen2.py:353
↓ 1 callers
Method
print
(self)
web/parms.py:95
↓ 1 callers
Function
print_batch
(batch, tokenizer, args)
tools/trainer_v4_48_3.py:315
↓ 1 callers
Function
print_grad_status
(model)
tools/finetune_sts_v4_48_3.py:622
↓ 1 callers
Method
process_anyres
(self, img_or_path)
vita_audio/data/processor/image_processor.py:239
↓ 1 callers
Method
process_dynamic
(self, img_or_path)
vita_audio/data/processor/image_processor.py:262
↓ 1 callers
Method
process_ret
(self, to_ret)
vita_audio/data/dataset_deepseek.py:149
↓ 1 callers
Method
process_ret
(self, to_ret)
vita_audio/data/dataset_hunyuan.py:124
↓ 1 callers
Method
process_ret
(self, to_ret)
vita_audio/data/dataset_cosyvoice2.py:123
↓ 1 callers
Method
process_ret
(self, to_ret)
vita_audio/data/dataset_llama3.py:123
↓ 1 callers
Method
process_ret
(self, to_ret)
vita_audio/data/dataset_qwen2.py:149
↓ 1 callers
Method
put
Add items to the buffer in a thread-safe manner. Parameters: - items (list or array-like): The items to be added to the buff
web/queue.py:29
↓ 1 callers
Function
resize_and_pad_image
Resize and pad an image to a target resolution while maintaining aspect ratio. Args: image (PIL.Image.Image): The input image.
vita_audio/data/processor/image_processor.py:325
↓ 1 callers
Function
select_best_resolution
Selects the best resolution from a list of possible resolutions based on the original size. Args: original_size (tuple): The origina
vita_audio/data/processor/image_processor.py:285
↓ 1 callers
Function
sequence_mask
(lengths, maxlen=None, dtype=torch.float32, device=None)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:358
↓ 1 callers
Function
shift_code
(codes, codebook_size, vq_strides)
tools/get_neural_audio_codecs.py:264
↓ 1 callers
Function
shift_code
(codes, codebook_size, vq_strides)
vita_audio/tokenizer_snac.py:150
↓ 1 callers
Method
show_statistic
(self)
vita_audio/data/dataset_deepseek.py:254
↓ 1 callers
Method
show_statistic
(self)
vita_audio/data/dataset_hunyuan.py:229
↓ 1 callers
Method
show_statistic
(self)
vita_audio/data/dataset_cosyvoice2.py:228
↓ 1 callers
Method
show_statistic
(self)
vita_audio/data/dataset_llama3.py:228
↓ 1 callers
Method
show_statistic
(self)
vita_audio/data/dataset_qwen2.py:254
↓ 1 callers
Function
stable_codec
()
tools/get_neural_audio_codecs.py:342
↓ 1 callers
Function
stripoff_tags
(x)
evaluation/compute-cer.py:50
↓ 1 callers
Function
stripoff_tags
(x)
evaluation/compute-wer.py:48
↓ 1 callers
Function
sts_stream_task
()
tools/inference_sts.py:846
↓ 1 callers
Function
sts_task
()
tools/inference_sts.py:824
↓ 1 callers
Function
text_audio_interval
(input_ids, AUD_START_ID, AUD_END_ID, text_audio_interval_ratio)
vita_audio/data/dataset_deepseek.py:907
↓ 1 callers
Function
text_audio_interval
(input_ids, AUD_START_ID, AUD_END_ID)
vita_audio/data/dataset_hunyuan.py:751
↓ 1 callers
Function
text_audio_interval
(input_ids, AUD_START_ID, AUD_END_ID, text_audio_interval_ratio)
vita_audio/data/processor/audio_processor.py:226
↓ 1 callers
Method
text_audio_interval
(self, content_input_id, AUD_START_ID, AUD_END_ID)
vita_audio/data/processor/audio_processor.py:77
↓ 1 callers
Function
text_stream_task
()
tools/inference_sts.py:802
↓ 1 callers
Function
text_task
()
tools/inference_sts.py:780
↓ 1 callers
Function
tts_stream_task
()
tools/inference_sts.py:963
↓ 1 callers
Function
tts_task
()
tools/inference_sts.py:904
↓ 1 callers
Function
update_tokenizer
(tokenizer)
vita_audio/tokenizer.py:22
↓ 1 callers
Function
update_tokenizer_for_cosyvoice2
(tokenizer)
vita_audio/tokenizer_cosyvoice2.py:34
↓ 1 callers
Function
update_tokenizer_for_glm4voice
(tokenizer)
vita_audio/tokenizer_glm4voice.py:45
↓ 1 callers
Function
update_tokenizer_for_s2s
(tokenizer, model_type)
vita_audio/tokenizer.py:47
↓ 1 callers
Function
update_tokenizer_for_sensevoice_glm4voice
(tokenizer)
vita_audio/tokenizer_sensevoice_glm4voice.py:50
↓ 1 callers
Function
update_tokenizer_for_sensevoice_sparktts
(tokenizer)
vita_audio/tokenizer_sensevoice_sparktts.py:37
↓ 1 callers
Function
update_tokenizer_for_snac
(tokenizer)
vita_audio/tokenizer_snac.py:36
↓ 1 callers
Function
usage
()
evaluation/compute-cer.py:294
↓ 1 callers
Function
usage
()
evaluation/compute-wer.py:287
Method
__call__
(self, instances: Sequence[Dict])
vita_audio/data/data_collator.py:22
Method
__call__
x: torch.Tensor audio chunk (see examples in repo) return_seconds: bool (default - False) whether return tim
web/vad.py:64
Method
__getitem__
(self, index)
vita_audio/data/dataset_deepseek.py:269
Method
__getitem__
(self, index)
vita_audio/data/dataset_hunyuan.py:244
Method
__getitem__
(self, index)
vita_audio/data/dataset_cosyvoice2.py:243
Method
__getitem__
(self, index)
vita_audio/data/dataset_llama3.py:243
Method
__getitem__
(self, index)
vita_audio/data/dataset_mistral.py:61
Method
__getitem__
(self, index)
vita_audio/data/dataset_qwen2.py:269
Method
__getitem__
(self, idx)
evaluation/evaluate_seedtts.py:91
Method
__getitem__
(self, idx)
evaluation/evaluate_asr.py:54
Method
__getitem__
(self, idx)
evaluation/evaluate_libritts.py:46
Method
__getitem__
(self, idx)
evaluation/evaluate_sqa.py:48
Method
__init__
( self, tokenizer: "AutoTokenizer", skip_prompt: bool = False, timeout: Option
web_demo_stream.py:163
Method
__init__
( self, tokenizer: "AutoTokenizer", skip_prompt: bool = False, timeout: Option
tools/inference_sts.py:72
Method
__init__
( self, tokenizer: "AutoTokenizer", skip_prompt: bool = False, timeout: Option
tools/inference_sts.py:129
Method
__init__
(self, model_path="VITA-MLLM/VITA-Audio-Plus-Boost", device='cuda',
Kimi-Audio-Evalkit/almeval/models/vita_audio.py:22
Method
__init__
(self, model_name_or_path, flow_path=None, rank=None)
vita_audio/tokenizer_sensevoice_glm4voice.py:84
Method
__init__
(self, model_name_or_path, flow_path=None, rank=None)
vita_audio/tokenizer_glm4voice.py:78
Method
__init__
(self, model_name_or_path, rank=None)
vita_audio/tokenizer_sensevoice_sparktts.py:71
Method
__init__
(self, model_name_or_path, rank=None)
vita_audio/tokenizer_cosyvoice2.py:67
Method
__init__
(self, model_name_or_path, rank=None)
vita_audio/tokenizer_snac.py:69
Method
__init__
( self, *args, **kwargs, )
vita_audio/data/dataset_deepseek.py:25
Method
__init__
( self, *args, **kwargs, )
vita_audio/data/dataset_hunyuan.py:25
Method
__init__
( self, cfg_path, tokenizer, image_size=448, image_token_length=1024,
vita_audio/data/dataset_base.py:27
Method
__init__
( self, *args, **kwargs, )
vita_audio/data/dataset_cosyvoice2.py:25
Method
__init__
( self, *args, **kwargs, )
vita_audio/data/dataset_llama3.py:25
Method
__init__
( self, *args, **kwargs, )
vita_audio/data/dataset_mistral.py:23
Method
__init__
( self, *args, **kwargs, )
vita_audio/data/dataset_qwen2.py:25
Method
__init__
( self, process_type, image_size=448, normalize_type="imagenet", min_p
vita_audio/data/processor/image_processor.py:21
Method
__init__
( self, audio_tokenizer_path=None, audio_tokenizer_type=None, text_audio_inter
vita_audio/data/processor/audio_processor.py:13
Method
__init__
(self, config: Qwen2Config, layer_idx: int)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:180
Method
__init__
Qwen2RMSNorm is equivalent to T5LayerNorm
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:254
Method
__init__
(self, config: Qwen2Config, layer_idx: int)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:274
Method
__init__
(self, config: Qwen2Config, device=None)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:331
Method
__init__
(self, config: Qwen2Config)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:524
Method
__init__
(self, config)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:838
Method
__init__
(self, config)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:1366
Method
__init__
(self, config)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:1462
Method
__init__
(self, config)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_qwen2.py:1552
Method
__init__
(self, proj_input_size, hidden_size)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/resampler_projector.py:9
Method
__init__
(self, config: Qwen2Config, layer_idx: int)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modular_qwen2.py:39
Method
__init__
(self, config: Qwen2Config, layer_idx: int)
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modular_qwen2.py:106
Method
__init__
( self, vocab_file=None, merges_file=None, tokenizer_file=None, unk_to
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/tokenization_qwen2_fast.py:83
Method
__init__
Construct an PositionwiseFeedForward object.
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:136
Method
__init__
Construct an MultiHeadedAttention object.
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:159
Method
__init__
Construct an EncoderLayer object.
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:371
Method
__init__
( self, input_size: int, output_size: int = 256, attention_heads: int = 4,
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:521
Method
__init__
( self, specaug: str = None, specaug_conf: dict = None, normalize: str = None,
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:664
Method
__init__
( self, config, )
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/modeling_sensevoice.py:1067
Method
__init__
( self, vocab_size=151936, hidden_size=4096, intermediate_size=22016,
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/configuration_qwen2.py:143
Method
__init__
( self, vocab_file, merges_file, errors="replace", unk_token="<|endoft
vita_audio/models/qwen2_mtp_sensevoice_v4_48_3/tokenization_qwen2.py:139
← previous
next →
201–300 of 566, ranked by callers