Functions — github.com/TencentARC/MindOmni

hub / github.com/TencentARC/MindOmni / functions

Functions516 in github.com/TencentARC/MindOmni

⨍ Functions 516◇ Types & classes 89 ↳ Endpoints 2

↓ 2 callersMethodunpatchify

x: (N, T, patch_size**2 * C) imgs: (N, H, W, C)

pretrain/model.py:274

↓ 2 callersFunctionupdate_ema

Step the EMA model towards the current model.

pretrain/utils.py:22

↓ 2 callersMethodvae_encode

(self, x, dtype)

pretrain/pipeline.py:119

↓ 2 callersFunctionvae_encode_list

(vae, x, weight_dtype)

pretrain/utils.py:153

↓ 2 callersFunctionzipngram

(text: list, ngram_size: int)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:641

↓ 2 callersFunctionzipngram

(text: list, ngram_size: int)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:653

↓ 1 callersMethod__init__

(self, data_source, repeat_count: int)

rl-postrain/src/open-r1-multimodal/src/open_r1/trainer/vllm_grpo_trainer.py:100

↓ 1 callersMethod__init__

(self, mllm, image_decoder, connector, vae, processor, mllm_processor, device: Union[str, torch.device] = None

src/mindomni.py:32

↓ 1 callersMethod_attempt_get_data_info

(self, index)

pretrain/train_helper/concatedataset.py:70

↓ 1 callersMethod_attempt_get_data_info

(self, index)

pretrain/train_helper/subdata.py:94

↓ 1 callersMethod_attempt_get_data_info

(self, index)

pretrain/train_helper/data.py:150

↓ 1 callersMethod_enable_gradient_checkpointing

Enables gradient checkpointing for the model.

rl-postrain/src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py:513

↓ 1 callersMethod_generate_and_score_completions

(self, inputs: dict[str, Union[torch.Tensor, Any]], model)

rl-postrain/src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py:586

↓ 1 callersMethod_init_rope

(self)

src/image_decoder/modeling_phi3.py:397

↓ 1 callersMethod_init_rope

(self)

pretrain/modeling_phi3.py:397

↓ 1 callersMethod_load_image

(self, image: Image.Image, input_size: int=448, max_num:int=12)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py:127

↓ 1 callersMethod_prepare

Prepare ._gts and ._dts for evaluation based on params :return: None

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/pycocotools/cocoeval.py:82

↓ 1 callersMethodadd_prefix_instruction

(self, prompt)

pretrain/processor.py:114

↓ 1 callersMethodadd_prefix_instruction_llm

(self, prompt, llm_processor, input_llm_images=None, input_images=None, input_images_shape=None, short_instruc

pretrain/processor.py:122

↓ 1 callersFunctionall_match_reward

(content, sol, **kwargs)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:776

↓ 1 callersFunctionall_match_reward

(content, sol, **kwargs)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:788

↓ 1 callersFunctionbuild_distilabel_pipeline

( model: str, base_url: str = "http://localhost:8000/v1", prompt_column: Optional[str] = None,

rl-postrain/src/open-r1-multimodal/src/open_r1/generate.py:22

↓ 1 callersFunctionbuild_gradio

()

app.py:54

↓ 1 callersFunctionbuild_message_edit

(edit_prompt, input_llm_images, think_content, max_input_image_size)

pretrain/train_helper/validate.py:107

↓ 1 callersFunctionbuild_transform

(input_size)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py:267

↓ 1 callersFunctioncalculate_map

(pred_bbox_list, gt_bbox_list, score_type=0)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:238

↓ 1 callersFunctioncalculate_map

(pred_bbox_list, gt_bbox_list, score_type=0)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:250

↓ 1 callersFunctioncompute_gen_kl_loss

(output_images, gt_prompt, messages, vae, processor, llm_processor, model_llm, model, weight_dtype, output_hid

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/gen_kl.py:78

↓ 1 callersFunctioncosine_reward

(content, tokenizer, acc_reward, **kwargs)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:565

↓ 1 callersFunctioncosine_reward

(content, tokenizer, acc_reward, **kwargs)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:577

↓ 1 callersMethodcreate_connector_position

(self, llm_2d_attention_mask)

src/image_decoder/processor.py:105

↓ 1 callersMethodcreate_llm_vae_position

(self, llm_vae_attention_mask, llm_2d_attention_mask, num_tokens_for_output_images, llm_image_sizes)

pretrain/processor.py:366

↓ 1 callersFunctioncreate_logger

Create a logger that writes to a log file and stdout.

pretrain/utils.py:7

↓ 1 callersMethodcreate_mask

(self, attention_mask, num_tokens_for_output_images)

src/image_decoder/processor.py:116

↓ 1 callersMethodcreate_model_card

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

rl-postrain/src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py:994

↓ 1 callersMethodcreate_position

(self, attention_mask, num_tokens_for_output_images)

src/image_decoder/processor.py:95

↓ 1 callersFunctioncrop_arr

(pil_image, max_image_size)

src/image_decoder/processor.py:12

↓ 1 callersFunctioncrop_arr

(pil_image, max_image_size)

pretrain/utils.py:64

↓ 1 callersMethodcrop_attention_mask_for_cache

(self, attention_mask, num_tokens_for_img)

src/image_decoder/scheduler.py:136

↓ 1 callersMethodcrop_position_ids_for_cache

(self, position_ids, num_tokens_for_img)

src/image_decoder/scheduler.py:128

↓ 1 callersFunctiondefault_accuracy_reward

(content, sol, **kwargs)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:781

↓ 1 callersFunctiondefault_accuracy_reward

(content, sol, **kwargs)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:793

↓ 1 callersMethoddisable_model_cpu_offload

(self)

src/image_decoder/image_pipeline.py:77

↓ 1 callersMethoddisable_model_cpu_offload

(self)

pretrain/pipeline.py:143

↓ 1 callersFunctiondynamic_preprocess

(image, min_num=1, max_num=12, image_size=448, use_thumbnail=False)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py:292

↓ 1 callersMethodenable_model_cpu_offload

(self)

src/image_decoder/image_pipeline.py:69

↓ 1 callersMethodenable_model_cpu_offload

(self)

pretrain/pipeline.py:133

↓ 1 callersFunctionevaluate_answer_similarity

Use llm to evaluate answer similarity.

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:162

↓ 1 callersFunctionevaluate_answer_similarity

Use llm to evaluate answer similarity.

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:174

↓ 1 callersMethodevict_previous_layer

Moves the previous layer cache to the CPU

src/image_decoder/transformer.py:27

↓ 1 callersMethodevict_previous_layer

Moves the previous layer cache to the CPU

pretrain/transformer.py:40

↓ 1 callersFunctionextract_problem_solution

(gpt4o_response)

rl-postrain/src/open-r1-multimodal/local_scripts/prepare_hf_data.py:32

↓ 1 callersFunctionextract_system_message

(conversation_list)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py:258

↓ 1 callersFunctionfind_closest_aspect_ratio

(aspect_ratio, target_ratios, width, height, image_size)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py:277

↓ 1 callersFunctionfix_a_slash_b

(string)

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/math.py:118

↓ 1 callersFunctionfix_fracs

(string)

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/math.py:86

↓ 1 callersFunctionfix_sqrt

(string)

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/math.py:143

↓ 1 callersMethodforward

(self, hidden_states: torch.FloatTensor)

src/image_decoder/modeling_phi3.py:338

↓ 1 callersMethodforward

(self, hidden_states: torch.FloatTensor)

pretrain/modeling_phi3.py:338

↓ 1 callersMethodfrom_pretrained

(cls, model_name, vae_path: str=None)

pretrain/pipeline.py:83

↓ 1 callersMethodfull_init

Fully initialize the dataset.

pretrain/train_helper/webdataset_laion.py:68

↓ 1 callersMethodfull_init

Fully initialize the dataset.

pretrain/train_helper/webdataset_.py:66

↓ 1 callersMethodgenerate_image

(self, height, width, guidance_scale, inference_steps, separate_cfg_infer, offload_model, seed, max_input_imag

src/mindomni.py:191

↓ 1 callersMethodgenerate_text

(self, text, input_llm_images, do_sample, temperature, max_new_tokens, only_understand)

src/mindomni.py:228

↓ 1 callersMethodgetCatIds

filtering parameters. default skips that filter. :param catNms (str array) : get cats for given cat names :param supNms (str

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/pycocotools/coco.py:114

↓ 1 callersFunctionget_2d_sincos_pos_embed

grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or [1+grid_size*grid_size, embed_dim] (w/ or

src/image_decoder/model.py:79

↓ 1 callersFunctionget_2d_sincos_pos_embed

grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or [1+grid_size*grid_size, embed_dim] (w/ or

pretrain/model.py:84

↓ 1 callersFunctionget_2d_sincos_pos_embed_from_grid

(embed_dim, grid)

src/image_decoder/model.py:99

↓ 1 callersFunctionget_2d_sincos_pos_embed_from_grid

(embed_dim, grid)

pretrain/model.py:104

↓ 1 callersFunctionget_callbacks

(train_config, model_config)

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/callbacks.py:79

↓ 1 callersMethodget_custom_multimodal_keywords

(self)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:33

↓ 1 callersMethodget_custom_processing_keywords

(self)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:41

↓ 1 callersMethodget_data_info

Get data info from the dataset.

pretrain/train_helper/webdataset_laion.py:48

↓ 1 callersMethodget_eos_token_id

(self, processing_class)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py:53

↓ 1 callersFunctionget_gpu_count_for_vllm

vLLM enforces a constraint that the number of attention heads must be divisible by the number of GPUs and 64 must be divisible by the number of GPUs.

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/hub.py:120

↓ 1 callersFunctionget_image_data_url

(image_input)

rl-postrain/src/open-r1-multimodal/local_scripts/create_vision_cot_data.py:47

↓ 1 callersMethodget_input_embeddings

(self)

pretrain/modeling_phi3.py:967

↓ 1 callersMethodget_model_class

(self, model_id: str, model_init_kwargs: dict)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:15

↓ 1 callersMethodget_non_generate_params

(self)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:37

↓ 1 callersMethodget_offlaod_layer

(self, layer_idx: int, device: torch.device)

src/image_decoder/transformer.py:33

↓ 1 callersMethodget_offlaod_layer

(self, layer_idx: int, device: torch.device)

pretrain/transformer.py:46

↓ 1 callersFunctionget_param_count_from_repo_id

Function to get model param counts from safetensors metadata or find patterns like 42m, 1.5b, 0.5m or products like 8x7b in a repo ID.

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/hub.py:88

↓ 1 callersMethodget_processing_class

(self)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:25

↓ 1 callersMethodget_vision_modules_keywords

(self)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:29

↓ 1 callersFunctionget_vlm_module

(model_name_or_path)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:919

↓ 1 callersFunctionget_vlm_module

(model_name_or_path)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_rec.py:198

↓ 1 callersFunctionget_vlm_module

(model_name_or_path)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:931

↓ 1 callersFunctiongpt4o_query

(image, prompt, max_retries=5, initial_delay=3)

rl-postrain/src/open-r1-multimodal/local_scripts/create_vision_cot_data.py:70

↓ 1 callersFunctionhas_answer_pattern

(text)

rl-postrain/src/open-r1-multimodal/local_scripts/prepare_hf_data.py:138

↓ 1 callersFunctionhas_empty_tags

(text)

rl-postrain/src/open-r1-multimodal/local_scripts/prepare_hf_data.py:132

↓ 1 callersFunctionhas_valid_image_size

(example)

rl-postrain/src/open-r1-multimodal/local_scripts/prepare_hf_data.py:144

↓ 1 callersFunctioninitialize_tokenizer

(model_path)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:58

↓ 1 callersFunctioninitialize_tokenizer

(model_path)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_ust.py:58

↓ 1 callersMethodinitialize_weights

(self)

src/image_decoder/model.py:217

↓ 1 callersMethodinitialize_weights

(self)

pretrain/model.py:241

↓ 1 callersFunctioniou

(box1, box2)

rl-postrain/src/open-r1-multimodal/src/open_r1/grpo_jsonl.py:418

↓ 1 callersMethodiou

(box1, box2)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/qwen_module.py:151

↓ 1 callersMethodis_embeds_input

(self)

rl-postrain/src/open-r1-multimodal/src/open_r1/vlm_modules/vlm_module.py:21

↓ 1 callersFunctionis_equiv

(str1, str2, verbose=False)

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/math.py:68

↓ 1 callersFunctionis_slurm_available

()

rl-postrain/src/open-r1-multimodal/src/open_r1/utils/callbacks.py:28

← previous next →101–200 of 516, ranked by callers