MCPcopy Create free account

hub / github.com/huggingface/smollm / functions

Functions1,907 in github.com/huggingface/smollm

⨍ Functions 1,907◇ Types & classes 810↳ Endpoints 7
↓ 2 callersFunctionread_json_file
(file)
vision/m4/testing_utils.py:867
↓ 2 callersMethodremove_empty_el_from_list
(list_)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:59
↓ 2 callersMethodremove_empty_el_from_list
(list_)
vision/m4/sourcing/data_collection/processors/pair_filtering.py:195
↓ 2 callersFunctionremove_extra_images
Removes images if there are more than `max_num_images`. Removes the associated image tokens from the text. Strategy: -Remove all ima
vision/m4/training/packing.py:69
↓ 2 callersMethodremove_non_printing_characters
(text, non_printing_characters_re)
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:63
↓ 2 callersFunctionrename_state_dict
(state_dict)
vision/m4/scripts/convert_zero_state_dict_for_new_siglip.py:13
↓ 2 callersFunctionretrieve_image
(url)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:216
↓ 2 callersFunctionrotate_half
Rotates half the hidden dims of the input.
vision/m4/models/vmistral/modeling_vmistral.py:308
↓ 2 callersFunctionrotate_half
Rotates half the hidden dims of the input.
vision/m4/models/idefics/modeling_idefics.py:141
↓ 2 callersFunctionrotate_half
Rotates half the hidden dims of the input.
vision/m4/models/vllama3/modeling_vllama3.py:302
↓ 2 callersFunctionrun_cmd
(cmd, check=True)
vision/m4/scripts/s3-upload-checkpoints.py:45
↓ 2 callersFunctionrun_cmd
(cmd, check=True)
vision/m4/scripts/s3_checkpoint_download_convert_upload.py:46
↓ 2 callersFunctionsample_clip_indices
(frames_per_clip: int, video_duration: float, sampling_fps: float, video_fps: float, max_clips: int)
vision/smolvlm2/smolvlm/datasets/dataset_clip_sampling.py:55
↓ 2 callersMethodsave_config_state
(self)
vision/m4/training/config.py:575
↓ 2 callersMethodsave_current_chat
Save the current chat to disk if it has any messages
tools/smol_tools/smol_tools/chatter.py:59
↓ 2 callersMethodsave_ideas_and_html_codes
(self)
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/02_01_generate_html_codes_prompt_1.py:135
↓ 2 callersMethodsave_ideas_and_html_codes
(self)
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/02_02_generate_html_codes_prompt_2.py:137
↓ 2 callersFunctionsave_image_caption_pair_example_in_files
(example, idx, saving_dir)
vision/m4/utils/datasets/create_webdataset_tar.py:81
↓ 2 callersFunctionsave_image_question_answer_triplet_example_in_files
(example, idx, saving_dir)
vision/m4/utils/datasets/create_webdataset_tar.py:111
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_343_smolvlm_05b_1st_stage/resize_embed_for_eou.py:56
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_341_smolvlm_025b_1st_stage/resize_embed_for_eou.py:56
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_347_smolvlm_500M_2nd_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_348_smolvlm_2B/resize_embed_for_eou.py:56
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_345_vsmollm2_256M_2nd_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_349_vsmollm2_500M_3rd_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_350_smolvlm_2B_2nd_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/experiments/pretraining/vloom/tr_346_vsmollm2_256M_3rd_stage/resize_embed_for_eou.py:44
↓ 2 callersFunctionsave_modified
(weight_name, new_weight)
vision/m4/scripts/resize_embed_for_eou.py:58
↓ 2 callersFunctionsave_sft_example_in_files
(example, idx, saving_dir)
vision/m4/utils/datasets/create_webdataset_tar.py:144
↓ 2 callersFunctionsave_split_ds
(split_dataset, split_name)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:275
↓ 2 callersFunctionsave_web_document_example_in_files
(example, idx, saving_dir)
vision/m4/utils/datasets/create_webdataset_tar.py:35
↓ 2 callersFunctionset_seed
Helper function for reproducible behavior to set the seed in ``random``, ``numpy``, ``torch`` Args: seed (:obj:`int`): The seed to s
vision/m4/testing_utils.py:210
↓ 2 callersFunctionshow_gpu_mem_util
(args)
vision/m4/evaluation/evaluators/in_contexter.py:30
↓ 2 callersMethodstandardize_whitespace
There are different whitespace characters.
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:67
↓ 2 callersMethodstrip
Way faster than text.strip(strip_characters) since strip_characters is a set instead of a str, and it contains a lot of elements (all
vision/m4/sourcing/data_collection/processors/web_document_filtering.py:104
↓ 2 callersFunctiontee
(line, sink, pipe, label="")
vision/m4/testing_utils.py:1080
↓ 2 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str,
vision/m4/models/idefics/configuration_idefics.py:296
↓ 2 callersFunctiontrain
Main fine-tuning entry point for your smolVLM model, with optional LoRA + bitsandbytes, and prints which submodules are frozen/unfrozen.
vision/smolvlm2/smolvlm/train/train.py:272
↓ 2 callersFunctiontransform
(img, scale_up_factor=None)
vision/m4/training/utils.py:364
↓ 2 callersMethodupdate_chat_display
(self, chat_display: tk.Text, new_text: str)
tools/smol_tools/demo_tkinter.py:688
↓ 2 callersMethodupdate_gas_and_gbs
Update m4, deepspeed and accelerate with the derived global_batch_size and grad_acc_size
vision/m4/training/trainer.py:350
↓ 2 callersFunctionupdate_model_kwargs_for_generation
(outputs, model_kwargs, is_encoder_decoder=False)
vision/m4/models/common.py:37
↓ 2 callersMethodupdate_state
(self, dataset_state)
vision/m4/training/dataset.py:1192
↓ 2 callersFunctionurls_to_images
(dataset, dataset_images, map_url_idx, num_proc, some_urls_are_already_retrieved=False)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:209
↓ 2 callersFunctionwebdoc_valid_sample
Check whether a sample is valid. :param sample: sample to be checked
vision/m4/training/dataset_utils.py:41
↓ 2 callersFunctionwrite_file
(path_file, to_write)
vision/m4/sourcing/data_collection/processors/web_document_extractor.py:24
↓ 2 callersFunctionwrite_json_file
Write a list of dictionaries into a JSON file with pretty indentation.
vision/smolvlm2/scripts/sort.py:108
↓ 1 callersFunctionGetInstruction
(inputs: StepInput)
text/data/smoltalk/magpie_ultra_v1/pipeline.py:394
↓ 1 callersMethod__get_builders__
(self)
vision/m4/sourcing/pmd/scripts/pmd.py:69
↓ 1 callersMethod__get_robots_txt__
(self, robots_txt_url)
vision/m4/sourcing/pmd/helpers.py:136
↓ 1 callersMethod__init__
( self, path_binary_classifier, path_processor_binary_classifier, )
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/05_binary_classification.py:90
↓ 1 callersMethod__init__
(self, config)
vision/smolvlm2/smolvlm/model/modeling_smollmm.py:17
↓ 1 callersMethod__init__
out_additional_features: int. Number of additional trainable dimensions. Only makes sense when `partially_freeze=True`. partially_fre
vision/m4/models/custom_modules.py:256
↓ 1 callersMethod__iter__
(self)
vision/m4/training/dataset.py:562
↓ 1 callersMethod_add_special_tokens_to_prompt
(self, prompt)
vision/m4/models/vgpt2/evaluation_perplexity_in_context_vgpt2.py:120
↓ 1 callersMethod_apply_sampling_strategy
(self, strategy: str)
vision/smolvlm2/smolvlm/datasets/dataset.py:475
↓ 1 callersMethod_apply_sampling_strategy
(self, strategy: str)
vision/smolvlm2/smolvlm/datasets/dataset_clip_sampling.py:607
↓ 1 callersMethod_attn
(self, query, key, value, attention_mask=None, head_mask=None)
vision/m4/models/vgpt2/modeling_vgpt2.py:182
↓ 1 callersMethod_call_tools
(self, tool_calls: List[Dict[str, Any]])
tools/smol_tools/smol_tools/agent.py:120
↓ 1 callersMethod_check_default_dict_in_train_logs
(self, train_logs)
vision/m4/training/trainer.py:1431
↓ 1 callersMethod_check_if_training_is_over
(self, curr_opt_step, max_num_updates)
vision/m4/training/trainer.py:1360
↓ 1 callersMethod_check_jz_time_and_memory
(self, curr_opt_step)
vision/m4/training/trainer.py:1223
↓ 1 callersMethod_check_kill_switch
(self)
vision/m4/training/trainer.py:1219
↓ 1 callersMethod_check_remaining_indices
When taking screenshots of some websites, it often fails for some reasons. Therefore, we do a try/except and skip the indices of the json wher
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/04_screenshot_html_codes.py:49
↓ 1 callersMethod_check_sigterm_signal
(self)
vision/m4/training/trainer.py:1244
↓ 1 callersMethod_compatibility_checks
(self)
vision/m4/training/trainer.py:360
↓ 1 callersMethod_compute_relaxed_vqa_accuracy
From https://aclanthology.org/2022.findings-acl.177.pdf We use a relaxed accuracy measure for the numeric answers to allow a minor in
vision/m4/evaluation/custom_metrics/open_ended_vqa_metrics.py:147
↓ 1 callersMethod_configure_optimizer_and_scheduler
defines model optimizer and lr scheduler
vision/m4/training/trainer.py:427
↓ 1 callersMethod_convert_llava_to_openai_format
(self, llava_entry: Dict[str, str])
vision/smolvlm2/smolvlm/datasets/dataset.py:673
↓ 1 callersMethod_convert_llava_to_openai_format
(self, llava_entry: Dict[str, str])
vision/smolvlm2/smolvlm/datasets/dataset_clip_sampling.py:805
↓ 1 callersMethod_convert_png_to_jpg
(self, bytes_image_png: bytes, jpg_quality: int)
vision/data/datasets_processing_scripts/build_websight_v02/python_scripts/04_screenshot_html_codes.py:86
↓ 1 callersFunction_convert_to_rgb
(image)
vision/data/datasets_processing_scripts/build_laion_coco_dataset/python_scripts/train_bin_classif.py:356
↓ 1 callersMethod_create_example_prompt
(self, prompt_template_id, caption)
vision/m4/models/vgpt2/evaluation_image_caption_matching_vgpt2.py:169
↓ 1 callersMethod_create_image_caption_pair_prompt
(self, caption="", context=None)
vision/m4/models/vgpt2/evaluation_perplexity_in_context_vgpt2.py:114
↓ 1 callersMethod_create_prefix_prompt
(self, prompt_template_id)
vision/m4/models/vgpt2/evaluation_image_caption_matching_vgpt2.py:187
↓ 1 callersMethod_create_prefix_prompt
(self, prompt_template_id)
vision/m4/models/vgpt2/evaluation_classification_in_context_vgpt2.py:422
↓ 1 callersMethod_create_prefix_prompt
(self, prompt_template_id)
vision/m4/models/vgpt2/evaluation_captioning_in_context_vgpt2.py:305
↓ 1 callersMethod_create_prefix_prompt
(self, prompt_template_id)
vision/m4/models/vgpt2/evaluation_open_ended_vqa_in_context_vgpt2.py:427
↓ 1 callersFunction_decode_pdf_pages
( sample, num_anno_pages, )
vision/m4/training/dataset_utils.py:235
↓ 1 callersMethod_do_validation
(self, progress, curr_opt_step)
vision/m4/training/trainer.py:1452
↓ 1 callersMethod_dynamic_frequency_update
dynamic RoPE layers should recompute `inv_freq` in the following situations: 1 - growing beyond the cached sequence length (allow sca
vision/m4/models/vllama3/modeling_vllama3.py:235
↓ 1 callersMethod_end_of_epoch_reset_train_logs
(self, train_logs)
vision/m4/training/trainer.py:1441
↓ 1 callersFunction_expand_mask
Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.
vision/m4/models/idefics/modeling_idefics.py:75
↓ 1 callersMethod_extract_images_not_in_simplified_dom_tree
(self, html_str, page_url, images_in_simplified_dom_tree)
vision/m4/sourcing/data_collection/processors/pair_extractor.py:61
↓ 1 callersMethod_extraction
(self, html_str, page_url, diff_level_paths_max_cutoff=5)
vision/m4/sourcing/data_collection/processors/pair_extractor.py:38
↓ 1 callersFunction_fetch_image
Load a single image (from local path, URL, or base64) and resize it via 'smart_resize' constraints.
vision/smolvlm2/smolvlm/mm_utils.py:168
↓ 1 callersFunction_fix_a_slash_b
(string)
text/evaluation/smollm2/math_utils.py:40
↓ 1 callersFunction_fix_fracs
(string)
text/evaluation/smollm2/math_utils.py:8
↓ 1 callersFunction_fix_sqrt
(string)
text/evaluation/smollm2/math_utils.py:57
↓ 1 callersMethod_flash_attention_forward
Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th
vision/m4/models/perceiver/perceiver.py:289
↓ 1 callersMethod_flash_attention_forward
Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th
vision/m4/models/vmistral/modeling_vmistral.py:600
↓ 1 callersMethod_flash_attention_forward
Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th
vision/m4/models/vllama3/modeling_vllama3.py:602
↓ 1 callersMethod_format_texts
(self, list_nodes)
vision/m4/sourcing/data_collection/processors/pre_extraction_simplificator.py:116
↓ 1 callersMethod_generate_examples
(self, files, paths)
vision/m4/sourcing/pmd/jz_loaders/jz_wit/jz_wit.py:83
↓ 1 callersMethod_generate_examples
(self, annotations_file)
vision/m4/sourcing/pmd/jz_loaders/jz_conceptual_captions/jz_conceptual_captions.py:137
↓ 1 callersFunction_get_default_logging_level
If M4_VERBOSITY env var is set to one of the valid choices return that as the new default level. If it is not - fall back to `_default_log_le
vision/m4/utils/logging.py:33
↓ 1 callersMethod_get_system_metrics_logs
(self, memory_value_max)
vision/m4/training/trainer.py:1971
↓ 1 callersFunction_get_unpad_data
(attention_mask)
vision/m4/models/perceiver/perceiver.py:34
↓ 1 callersFunction_get_unpad_data
(attention_mask)
vision/m4/models/vmistral/modeling_vmistral.py:258
↓ 1 callersFunction_get_unpad_data
(attention_mask)
vision/m4/models/vllama3/modeling_vllama3.py:158
← previousnext →301–400 of 1,907, ranked by callers