MCPcopy Create free account

hub / github.com/huggingface/smollm / functions

Functions1,907 in github.com/huggingface/smollm

↓ 1 callersFunctionprocess_document
(text: str)
vision/m4/sourcing/processing/extracting_ngrams/extract_documents_ngrams.py:27
↓ 1 callersFunctionprocess_html_css
(html_css_file)
vision/data/datasets_processing_scripts/build_image_website_code/03_extraction_html_css_codes.py:10
↓ 1 callersMethodprocess_ideas_websites
Keep only the main business idea, remove the details about the design of the website
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/02_02_generate_html_codes_prompt_2.py:78
↓ 1 callersMethodprocess_image
(self, image_bytes)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/07_01_nsfw_image_filtering.py:117
↓ 1 callersMethodprocess_inputs_binary_classification
(self, texts, images)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/05_binary_classification.py:115
↓ 1 callersFunctionprocess_shard
(shard_id, data_dir)
vision/data/datasets_processing_scripts/enwiki/python_scripts/02_load_wit_images_in_ds.py:95
↓ 1 callersFunctionprocess_yaml
Process the YAML file, count samples, and validate paths.
vision/smolvlm2/scripts/mixture_checker.py:42
↓ 1 callersMethodprune_heads
(self, heads)
vision/m4/models/vgpt2/modeling_vgpt2.py:167
↓ 1 callersFunctionpynmvl_handle
(accelerator)
vision/m4/training/utils.py:578
↓ 1 callersFunctionremove_duplicated_images
(texts, images, metadata)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/13_final_processing.py:100
↓ 1 callersFunctionremove_end_END_OF_DOCUMENT_TOKEN_TO_BE_REPLACED
(texts, images, metadata)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/13_final_processing.py:168
↓ 1 callersMethodremove_non_printing_characters
(text, non_printing_characters_re)
vision/m4/sourcing/data_collection/processors/pair_filtering.py:199
↓ 1 callersFunctionremove_spam_paragraphs
(texts, images, metadata)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/13_final_processing.py:129
↓ 1 callersFunctionremove_texts_only_END_OF_DOCUMENT_TOKEN_TO_BE_REPLACED
(example)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/13_final_processing.py:236
↓ 1 callersFunctionrename_state_dict
(state_dict)
vision/m4/scripts/convert_vmistral_old_siglip_to_new_siglip.py:12
↓ 1 callersMethodreplace_digits_with_zeros
(text, digits_re)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:360
↓ 1 callersMethodreplace_overused_color_from_html_code
(self, html_code)
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/02_01_generate_html_codes_prompt_1.py:130
↓ 1 callersMethodreplace_overused_color_from_html_code
(self, html_code)
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/02_02_generate_html_codes_prompt_2.py:132
↓ 1 callersMethodreplace_unicode_punctuation
(text, unicode_punctuation)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:364
↓ 1 callersMethodreset_jsonl_stats
(self)
vision/m4/utils/activation_tracker.py:171
↓ 1 callersMethodreset_state
(self)
vision/m4/training/dataset.py:1344
↓ 1 callersMethodresize_and_center_crop
(self, image: Image.Image, target_size: int)
tools/smolvlm_local_inference/SmolVLM_video_inference.py:16
↓ 1 callersFunctionretrieve_image
(url)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/04_merge_web_docs_with_images.py:71
↓ 1 callersFunctionrun_cmd
(cmd, check=True)
vision/m4/scripts/convert-checkpoints.py:35
↓ 1 callersFunctionrun_conv
(text: str)
vision/m4/sourcing/processing/app.py:48
↓ 1 callersFunctionrun_query
(ngram_ids: List[int])
vision/m4/sourcing/processing/app.py:55
↓ 1 callersFunctionsample_has_all_files
(current_sample)
vision/m4/training/dataset_utils.py:55
↓ 1 callersMethodsave_commit_hash
(self)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:427
↓ 1 callersMethodsave_dataset
(self)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:422
↓ 1 callersFunctionsave_image_caption_pair_example_in_files_with_num_shards
( example, idx, saving_dir, num_examples_per_shard, save_shard_prefix )
vision/m4/utils/datasets/create_webdataset_tar.py:101
↓ 1 callersFunctionsave_image_question_answer_triplet_example_in_files_with_num_shards
( example, idx, saving_dir, num_examples_per_shard, save_shard_prefix )
vision/m4/utils/datasets/create_webdataset_tar.py:134
↓ 1 callersFunctionsave_model_and_config
(new_checkpoint_path, new_model)
vision/m4/scripts/convert_vmistral_old_siglip_to_new_siglip.py:25
↓ 1 callersMethodsave_new_ds
(self)
vision/data/datasets_processing_scripts/build_concatenation_datasets_sft/merge_on_image_individual_dataset.py:130
↓ 1 callersFunctionsave_sft_example_in_files_with_num_shards
(example, idx, saving_dir, num_examples_per_shard, save_shard_prefix)
vision/m4/utils/datasets/create_webdataset_tar.py:179
↓ 1 callersFunctionsave_split_sharded_already_splitted_dataset
(dataset, path_save_dir_sharded_dataset, shard_size)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:274
↓ 1 callersFunctionsave_split_sharded_dataset
(dataset, path_save_dir_sharded_dataset, shard_size)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:299
↓ 1 callersMethodsave_split_sharded_dataset
(self, reload_files=False)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:434
↓ 1 callersFunctionsave_web_document_example_in_files_with_num_shards
( example, idx, saving_dir, num_examples_per_shard, save_shard_prefix )
vision/m4/utils/datasets/create_webdataset_tar.py:71
↓ 1 callersMethodselect_mode
(self)
vision/m4/sourcing/data_collection/visualization/web_document_and_filtering_visualization.py:665
↓ 1 callersMethodset_title
(self)
vision/m4/sourcing/data_collection/visualization/web_document_visualization.py:18
↓ 1 callersMethodset_title
(self)
vision/m4/sourcing/data_collection/visualization/web_document_and_filtering_visualization.py:52
↓ 1 callersFunctionset_trainable_params
Freezes all parameters, then selectively unfreezes based on user flags: - tune_vision_tower => unfreeze vision tower - tune_mm_connecto
vision/smolvlm2/smolvlm/train/train.py:76
↓ 1 callersMethodsetup_batch_size_related_configs
batch_size-related configs are processed here. All this work is done here because it requires knowing the value of num_processes
vision/m4/training/trainer.py:242
↓ 1 callersFunctionshould_process
Heuristics to decide whether to upload this opt_step-XXX checkpoint or not
vision/m4/scripts/s3-upload-checkpoints.py:76
↓ 1 callersFunctionshould_process
Heuristics to decide whether to convert this opt_step-XXX checkpoint or not
vision/m4/scripts/convert-checkpoints.py:62
↓ 1 callersFunctionshould_process
Heuristics to decide whether to cleanup this opt_step-XXX checkpoint or not
vision/m4/scripts/cleanup-checkpoints.py:71
↓ 1 callersMethodshow_draft_input
(self, summary_x, summary_y, summary_width)
tools/smol_tools/demo_tkinter.py:272
↓ 1 callersFunctionsimilarity_score
(a_ij, o_q_i, tau=0.5)
vision/m4/evaluation/custom_metrics/doc_vqa_metrics.py:32
↓ 1 callersMethodsimplification_mode
(self)
vision/data/datasets_processing_scripts/clean_m4_prelimenary_experiments/explore/global_visualization.py:153
↓ 1 callersMethodsimplification_mode
(self)
vision/m4/sourcing/data_collection/visualization/global_visualization.py:124
↓ 1 callersMethodsimplification_mode
(self)
vision/m4/sourcing/data_collection/visualization/wikipedia/global_visualization.py:123
↓ 1 callersFunctionsplit_batch
(batch, chunk_size)
vision/m4/evaluation/utils.py:74
↓ 1 callersMethodsplit_on_whitespace
This method also removes concatenated spaces.
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:91
↓ 1 callersMethodstart_new_chat
(self, chat_display)
tools/smol_tools/demo_tkinter.py:598
↓ 1 callersMethodstate_dict
(self)
vision/m4/training/dataset.py:1388
↓ 1 callersMethodstatistics_without_filtering
(self)
vision/m4/sourcing/data_collection/visualization/pair_visualization.py:151
↓ 1 callersMethodstop
(self)
vision/m4/utils/training/timer.py:60
↓ 1 callersFunctionstrip_string
(string, skip_unit=False)
text/evaluation/smollm2/math_utils.py:209
↓ 1 callersFunctionsummary_failures_short
(tr)
vision/m4/testing_utils.py:970
↓ 1 callersMethodtokenization
(text, sentencepiece_model, join_on_whitespace)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:399
↓ 1 callersMethodtokenize_and_count_tokens
(self, list_texts: List[str])
vision/data/datasets_processing_scripts/build_concatenation_datasets_sft/merge_on_image_individual_dataset.py:146
↓ 1 callersFunctiontokenize_id
Tokenize the id string on common delimiters: hyphens, underscores, and periods. Return a set of non-empty tokens. For example: "
vision/smolvlm2/scripts/sort.py:28
↓ 1 callersFunctiontrainer_save_model_safe
Safely saves the model if not in DeepSpeed ZeRO stage-3.
vision/smolvlm2/smolvlm/train/train.py:50
↓ 1 callersFunctiontransform_img
(img)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/07_02_nsfw_image_visualization.py:68
↓ 1 callersFunctiontransform_img
(img)
vision/m4/sourcing/data_collection/visualization/choose_filtering_parameters_web_documents_node_level.py:216
↓ 1 callersFunctiontransform_img
(img)
vision/m4/sourcing/data_collection/visualization/get_stats_vis_choose_filtering_parameters_laion_pairs.py:52
↓ 1 callersFunctiontranslate_nested_expression
(expr)
vision/data/datasets_processing_scripts/build_concatenation_datasets_sft/build_ds_sft.py:3982
↓ 1 callersFunctiontranslate_sequential_expressions
(input_str)
vision/data/datasets_processing_scripts/create_fine_tuning_datasets/create_table_datasets.py:669
↓ 1 callersFunctionunroll_list
(list_)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/08_01_prepare_urldedup.py:25
↓ 1 callersFunctionunwrap_exception
(wrapped_function)
vision/m4/sourcing/pmd/helpers.py:92
↓ 1 callersMethodupdate_agent_output
(self, text_widget, new_text)
tools/smol_tools/demo_tkinter.py:499
↓ 1 callersMethodupdate_before_collapsing_meta
(self, batch: Dict)
vision/m4/sourcing/pmd/loader_builder.py:33
↓ 1 callersFunctionupdate_df_metrics_and_lists_for_extraction_method
_summary_ Given a Media_Node and the Extraction_Method_Name used to get this Media_Node, this function uses the Media_Node's values to update
vision/m4/sourcing/data_collection/visualization/pair_stat_dashboard.py:94
↓ 1 callersMethodupdate_improved_text
(self, text_widget, new_text)
tools/smol_tools/demo_tkinter.py:441
↓ 1 callersFunctionupdate_model_kwargs_for_generation
(outputs, model_kwargs)
vision/m4/models/vmistral/modeling_vmistral.py:197
↓ 1 callersFunctionupdate_sampling_strategies
Update each dataset entry's sampling_strategy according to these rules: - If always_full applies, use "random:100%". - Otherwise, com
vision/smolvlm2/scripts/create_mixture.py:123
↓ 1 callersFunctionurls_to_images
(web_document_dataset_without_images, image_dataset, map_url_idx, url_ban_words, num_proc)
vision/data/datasets_processing_scripts/build_webdocs_dataset/python_scripts/04_merge_web_docs_with_images.py:70
↓ 1 callersMethodurls_to_images
(self, reload_files=False)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:396
↓ 1 callersFunctionvalidate_media_paths
Validate paths for a sample of elements from the dataset. Returns percentage of failed paths.
vision/smolvlm2/scripts/mixture_checker.py:7
↓ 1 callersFunctionvalidate_optim_states_are_reset
for a new or fully reset optimizer we expect all zeros `exp_avg` and `exp_avg_sq` state tensors and step=1
vision/m4/training/debug_utils.py:21
↓ 1 callersMethodvisualization
(self)
vision/data/datasets_processing_scripts/clean_m4_prelimenary_experiments/explore/global_visualization.py:109
↓ 1 callersMethodvisualization
(self)
vision/m4/sourcing/data_collection/visualization/global_visualization.py:80
↓ 1 callersMethodvisualization
(self)
vision/m4/sourcing/data_collection/visualization/web_document_visualization.py:12
↓ 1 callersMethodvisualization
(self)
vision/m4/sourcing/data_collection/visualization/web_document_and_filtering_visualization.py:44
↓ 1 callersMethodvisualization
(self)
vision/m4/sourcing/data_collection/visualization/pair_visualization.py:22
↓ 1 callersMethodvisualization
(self)
vision/m4/sourcing/data_collection/visualization/wikipedia/global_visualization.py:79
↓ 1 callersMethodvisualize
(self)
vision/m4/sourcing/data_collection/visualization/pair_stat_dashboard.py:267
↓ 1 callersFunctionvqa_normalize_text
Process a text Source: https://github.com/GT-Vision-Lab/VQA/blob/master/PythonEvaluationTools/vqaEvaluation/vqaEval.py 1. Conversion of charac
vision/m4/evaluation/custom_metrics/utils.py:150
↓ 1 callersMethodvqa_normalize_text
(self, text)
vision/m4/evaluation/custom_metrics/utils.py:373
↓ 1 callersFunctionwrap_exception
(unwrapped_function)
vision/m4/sourcing/pmd/helpers.py:83
↓ 1 callersFunctionwrite_yaml
(data, file_path)
vision/smolvlm2/scripts/create_mixture.py:115
FunctionExtendSysPath
Temporary add given path to `sys.path`. Usage :: with ExtendSysPath('/path/to/dir'): mymodule = importlib.import_module('
vision/m4/testing_utils.py:527
Method_DATASETS_NAME
(self)
vision/m4/sourcing/pmd/loader_builder.py:38
Method_DATASETS_NAME
(self)
vision/m4/sourcing/pmd/loader_builder.py:118
Method__call__
(self, example)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/02_02_remove_opt_out.py:51
Method__call__
(self, example)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/make_laion_coco_1_4_dataset.py:47
Method__call__
(self, example)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/make_laion_coco_1_4_dataset.py:68
Method__call__
(self, example)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/make_laion_coco.py:46
Method__call__
(self, example)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/make_laion_coco.py:67
Method__call__
(self, batch)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/05_binary_classification.py:104
← previousnext →901–1,000 of 1,907, ranked by callers