MCPcopy Create free account

hub / github.com/huggingface/smollm / functions

Functions1,907 in github.com/huggingface/smollm

⨍ Functions 1,907◇ Types & classes 810↳ Endpoints 7
↓ 2 callersMethod_reset_train_logs
(self, train_logs)
vision/m4/training/trainer.py:1416
↓ 2 callersMethod_set_cos_sin_cache
(self, seq_len, device, dtype)
vision/m4/models/vmistral/modeling_vmistral.py:286
↓ 2 callersFunction_smart_nframes
Decide how many frames to pick from a video based on either: - 'nframes' in config - or 'fps' in config (or default_fps if none spe
vision/smolvlm2/smolvlm/mm_utils.py:72
↓ 2 callersMethodadd_task
(self, description, total, *args, **kwargs)
vision/m4/utils/progress.py:76
↓ 2 callersMethodapply
(self, img, **params)
vision/m4/training/utils.py:185
↓ 2 callersFunctionapply_print_resets
(buf)
vision/m4/testing_utils.py:341
↓ 2 callersFunctionapply_rotary_pos_emb
(q, k, cos, sin, position_ids)
vision/m4/models/vmistral/modeling_vmistral.py:316
↓ 2 callersFunctioncached_path
Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and retur
vision/m4/sourcing/pmd/cache_path.py:41
↓ 2 callersMethodcheck_character_repetition_ratio
( text, character_repetition_length, character_repetition_max_cutoff, )
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:172
↓ 2 callersMethodcheck_compatibilities
(self)
vision/m4/models/vgpt2/configuration_vgpt2.py:255
↓ 2 callersMethodcheck_flagged_word_ratio
( text, strip_characters, flagged_words, flagged_word_ratio_max_cutoff, )
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:260
↓ 2 callersMethodcheck_lang_id
(text, lang_id_model, target_lang_id, lang_id_min_cutoff)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:351
↓ 2 callersMethodcheck_number_words
(text, strip_characters, number_words_min_cutoff, number_words_max_cutoff)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:142
↓ 2 callersMethodcheck_perplexity_score
( text, non_printing_characters_re, digits_re, unicode_punctuation, se
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:435
↓ 2 callersMethodcheck_punctuation_ratio
( text, punctuation, punctuation_ratio_min_cutoff, min_nb_words=-1, )
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:301
↓ 2 callersFunctioncheck_s3_directory
(directory_path)
vision/m4/scripts/s3_checkpoint_download_convert_upload.py:19
↓ 2 callersMethodcheck_special_character_ratio
(text, special_characters, special_character_ratio_max_cutoff)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:222
↓ 2 callersMethodcheck_stopword_ratio
(text, strip_characters, stopwords, stopword_ratio_min_cutoff)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:241
↓ 2 callersFunctioncheck_valid_tokenizer
Check if the special tokens were correctly added to the tokenizer, and if they are not normalized.
vision/m4/utils/check_valid_tokenizer.py:1
↓ 2 callersFunctioncollapse_columns_to_meta
( batch: Dict, columns_to_collapse: List[str], meta_column_name: str, )
vision/m4/sourcing/pmd/helpers.py:32
↓ 2 callersFunctioncompute_cache_path
(cache_dir, hash)
vision/m4/sourcing/pmd/helpers.py:73
↓ 2 callersFunctioncompute_linear_tflops_per_batch_per_gpu
( batch_size, seq_len, in_features, out_features, count_backward=False, use_grad_check
vision/m4/training/utils.py:656
↓ 2 callersMethodcompute_punctuation_ratio
(text, punctuation, min_nb_words=-1)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:276
↓ 2 callersMethodcompute_score
(self, generated_captions, reference_captions)
vision/m4/evaluation/custom_metrics/unfolded_image_captioning_metrics.py:56
↓ 2 callersFunctionconvert_to_list
(obj)
vision/m4/evaluation/custom_metrics/unfolded_image_captioning_metrics.py:203
↓ 2 callersMethodcreate_dataset_images
(self)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:387
↓ 2 callersFunctioncreate_dataset_images_from_tar
( tar_paths, path_save_dir_tmp_datasets_images, num_proc, path_save_file_map_url_idx, path
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:131
↓ 2 callersFunctioncreate_dict_qbench
`split` is "train", "validation" or "test".
vision/data/datasets_processing_scripts/integrate_evaluation_benchmarks_chatbot/qbench.py:39
↓ 2 callersFunctioncreate_ds_mmbench
`split` is "validation" or "test".
vision/data/datasets_processing_scripts/integrate_evaluation_benchmarks_chatbot/mmbench_no_mcq.py:41
↓ 2 callersFunctioncreate_sample_single_label_dataset
(task, split)
vision/m4/evaluation/scripts/create_sample_evaluation_datasets.py:71
↓ 2 callersMethoddisplay_chat_history
(self, chat_display: tk.Text)
tools/smol_tools/demo_tkinter.py:694
↓ 2 callersFunctiondownload_images
( path_save_file_image_urls, path_save_dir_downloaded_images, number_sample_per_shard, image_s
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:109
↓ 2 callersMethoddownload_images
(self)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:376
↓ 2 callersFunctionexpand_inputs_for_generation
( input_ids, expand_size=1, is_encoder_decoder=False, attention_mask=None, encoder_outputs
vision/m4/models/common.py:6
↓ 2 callersFunctionfilter_out_tags
(tags)
vision/m4/evaluation/scripts/sync_evaluations_on_wandb.py:48
↓ 2 callersFunctionfinal_cleaning_node_level
(texts, images, metadata)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/13_final_processing.py:180
↓ 2 callersFunctionfind_positions
(array_to_check, item_to_find)
vision/m4/evaluation/custom_metrics/unfolded_classification_metrics.py:246
↓ 2 callersMethodformat_jsonl_logs
Similar to format_print_logs but for jsonl logs
vision/m4/training/trainer.py:1026
↓ 2 callersMethodformat_train_logs
(self, train_logs, logger_type=LoggingTypes.PRINT)
vision/m4/training/trainer.py:1063
↓ 2 callersMethodformat_val_logs
(self, val_logs, logger_type=LoggingTypes.PRINT)
vision/m4/training/trainer.py:1048
↓ 2 callersFunctionformat_value_to_list
(value)
vision/data/datasets_processing_scripts/create_evaluation_datasets/create_mmmu.py:62
↓ 2 callersMethodfreeze_relevant_params
(self, config=None)
vision/m4/models/vmistral/modeling_vmistral.py:1054
↓ 2 callersMethodfreeze_relevant_params
(self, config=None)
vision/m4/models/vgpt2/modeling_vgpt2.py:770
↓ 2 callersMethodfreeze_relevant_params
(self, config=None)
vision/m4/models/idefics/modeling_idefics.py:731
↓ 2 callersMethodfreeze_relevant_params
(self, config=None)
vision/m4/models/vllama3/modeling_vllama3.py:1147
↓ 2 callersMethodgather_metrics
Collating all metrics to gather into ONE call to `torch.distributed.all_gather` instead of doing one per metric x dataset_name.
vision/m4/training/trainer.py:796
↓ 2 callersMethodget_bar_chart_from_aggregate_metrics
( self, metrics_values, metrics_names, x_label_categories, x_label, y_label, bar_size=30 )
vision/m4/sourcing/data_collection/visualization/pair_stat_dashboard.py:352
↓ 2 callersMethodget_batch_metadata_columns
(batch: Dict)
vision/m4/sourcing/pmd/loader_builder.py:63
↓ 2 callersMethodget_chart_infos_from_3D_df
(self, df, col_list, rows_list, reduction="mean")
vision/m4/sourcing/data_collection/visualization/pair_stat_dashboard.py:366
↓ 2 callersMethodget_chat_history
(self)
tools/smol_tools/smol_tools/chatter.py:147
↓ 2 callersFunctionget_config
(print_config: bool = True)
vision/m4/training/config.py:585
↓ 2 callersFunctionget_config
(args=None)
vision/m4/evaluation/config.py:146
↓ 2 callersFunctionget_dataloader_from_config
( tokenizer, image_transforms, image_seq_len, seed, config, is_train, rank=None,
vision/m4/training/dataset.py:507
↓ 2 callersFunctionget_dataloaders
( config: Parameters, rank: int, world_size: int, tokenizer, train_image_transforms, v
vision/m4/training/dataset.py:156
↓ 2 callersMethodget_dist_infos_from_3D_df
(self, df, col_list, rows_list, max_value=None)
vision/m4/sourcing/data_collection/visualization/pair_stat_dashboard.py:378
↓ 2 callersMethodget_dom_viz_html
(self, html)
vision/data/datasets_processing_scripts/clean_m4_prelimenary_experiments/explore/global_visualization.py:143
↓ 2 callersFunctionget_finepersonas_linkedin_posts
( n: int | None = None, system_prompt: str = REWRITE_CONCISE_SYSTEM_PROMPT )
text/data/smoltalk/rewrite/pipeline/dataset.py:75
↓ 2 callersMethodget_image_urls
(self, batch)
vision/m4/sourcing/pmd/loader_builder.py:121
↓ 2 callersMethodget_input_embeddings
(self)
vision/m4/models/vmistral/modeling_vmistral.py:1397
↓ 2 callersMethodget_input_embeddings
(self)
vision/m4/models/vllama3/modeling_vllama3.py:1533
↓ 2 callersFunctionget_media_src
(node)
vision/m4/sourcing/data_collection/utils/simplification_utils.py:41
↓ 2 callersFunctionget_prompt_template_id
(args, task)
vision/m4/evaluation/utils.py:21
↓ 2 callersMethodget_saved_chats
Get list of saved chat IDs
tools/smol_tools/smol_tools/chatter.py:107
↓ 2 callersFunctionget_webdataset
( urls, ds_type: DatasetTypes, batch_size: int, shuffle_initial_urls_list, shuffle_before_
vision/m4/training/dataset_utils.py:385
↓ 2 callersFunctioninit_new
(matrix, matrix_additional)
vision/experiments/pretraining/vloom/tr_347_smolvlm_500M_2nd_stage/resize_embed_for_eou.py:41
↓ 2 callersFunctioninit_new
(matrix, matrix_additional)
vision/experiments/pretraining/vloom/tr_345_vsmollm2_256M_2nd_stage/resize_embed_for_eou.py:41
↓ 2 callersFunctioninit_new
(matrix, matrix_additional)
vision/experiments/pretraining/vloom/tr_349_vsmollm2_500M_3rd_stage/resize_embed_for_eou.py:41
↓ 2 callersFunctioninit_new
(matrix, matrix_additional)
vision/experiments/pretraining/vloom/tr_350_smolvlm_2B_2nd_stage/resize_embed_for_eou.py:41
↓ 2 callersFunctioninit_new
(matrix, matrix_additional)
vision/experiments/pretraining/vloom/tr_346_vsmollm2_256M_3rd_stage/resize_embed_for_eou.py:41
↓ 2 callersFunctioninstantiate_scorers
(metrics, scorers_map)
vision/m4/evaluation/custom_metrics/unfolded_image_captioning_metrics.py:83
↓ 2 callersFunctionis_accelerate_initialized
()
vision/m4/training/utils.py:802
↓ 2 callersFunctionis_bnb_available
()
vision/m4/testing_utils.py:186
↓ 2 callersFunctionis_deepspeed_used
()
vision/m4/training/utils.py:752
↓ 2 callersMethodis_image_file
(self, filename)
vision/smolvlm2/smolvlm/conversation.py:244
↓ 2 callersFunctionload
(module: torch.nn.Module, state_dict, prefix="")
vision/m4/training/utils.py:908
↓ 2 callersFunctionload_and_save_dataset
(task, split, save_dir)
vision/m4/evaluation/scripts/copy_remote_sample_datasets.py:22
↓ 2 callersFunctionload_annotations
(path_to_generation_data, target_task)
vision/m4/evaluation/scripts/mmmu_submission_format.py:20
↓ 2 callersFunctionload_annotations
(path_to_generation_data)
vision/m4/evaluation/scripts/visualize_generations.py:32
↓ 2 callersFunctionload_examples
(num_docs, css_rule)
vision/data/datasets_processing_scripts/clean_m4_prelimenary_experiments/explore/explore.py:32
↓ 2 callersFunctionload_hf_dataset
(dataset_path)
vision/m4/training/dataset.py:195
↓ 2 callersMethodload_resume_states
(self, resumable_states)
vision/m4/training/dataset.py:1195
↓ 2 callersMethodload_to_be_deduplicated
(self)
vision/m4/sourcing/data_collection/processors/web_document_image_deduplication.py:273
↓ 2 callersMethodload_web_document_dataset_train
(self)
vision/m4/sourcing/data_collection/processors/web_document_image_deduplication.py:260
↓ 2 callersFunctionmem_usage_formatted
(logging_type=LoggingTypes.PRINT)
vision/m4/training/utils.py:726
↓ 2 callersFunctionnon_empty_els_from_list
(list_)
vision/m4/sourcing/data_collection/visualization/choose_filtering_parameters_web_documents_node_level.py:26
↓ 2 callersFunctionpad_to_multiple_of_128
(tensor, embedding_matrix)
vision/experiments/pretraining/vloom/tr_343_smolvlm_05b_1st_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionpad_to_multiple_of_128
(tensor, embedding_matrix)
vision/experiments/pretraining/vloom/tr_341_smolvlm_025b_1st_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionpad_to_multiple_of_128
(tensor, embedding_matrix)
vision/experiments/pretraining/vloom/tr_348_smolvlm_2B/resize_embed_for_eou.py:44
↓ 2 callersFunctionpad_to_multiple_of_128
(tensor, embedding_matrix)
vision/m4/scripts/resize_embed_for_eou.py:46
↓ 2 callersFunctionparse_math_answer
(text: str, data_name)
text/evaluation/smollm2/math_utils.py:420
↓ 2 callersMethodperceptual_hashing
(image_dataset, num_proc)
vision/m4/sourcing/data_collection/processors/image_deduplicator.py:9
↓ 2 callersFunctionprepare_inputs_for_generation
(input_ids, past=None, **kwargs)
vision/m4/models/common.py:67
↓ 2 callersFunctionpreprocess
Comes from AiHarness
text/evaluation/smollm2/tasks.py:12
↓ 2 callersMethodprocess_chat_message
(self, message: str, chat_display: tk.Text)
tools/smol_tools/demo_tkinter.py:637
↓ 2 callersFunctionprocess_ds
(ds: Dataset, batch_size: int, num_proc: int)
vision/m4/utils/datasets/get_self_contained_ds.py:68
↓ 2 callersMethodprocess_image
(self, image, image_process_mode, return_pil=False, image_format="PNG")
vision/smolvlm2/smolvlm/conversation.py:183
↓ 2 callersFunctionprocess_json_file
Load a JSON file (expected to be a list of dictionaries). For each entry, compute the shared_id and store it in "shared_id". Also print s
vision/smolvlm2/scripts/sort.py:84
↓ 2 callersMethodprocess_summary_question
Process a follow-up question about the summarized text
tools/smol_tools/demo_tkinter.py:219
↓ 2 callersMethodprocess_text
(self, text, keep_only_first_word=False)
vision/m4/evaluation/custom_metrics/unfolded_image_captioning_metrics.py:38
↓ 2 callersFunctionpynvml_get_total_energy_in_joules
(handle)
vision/m4/training/utils.py:586
← previousnext →201–300 of 1,907, ranked by callers