MCPcopy Create free account

hub / github.com/TRI-ML/vlm-evaluation / functions

Functions337 in github.com/TRI-ML/vlm-evaluation

⨍ Functions 337◇ Types & classes 99↳ Endpoints 9
↓ 42 callersMethodload
Loads model using a combination of `transformers.AutoModelForCausalLM` along with the special `LlavaLlamaForCausalLM` class defined i
vlm_eval/models/llava.py:113
↓ 16 callersFunctioninitialize_overwatch
(name: str)
vlm_eval/overwatch/overwatch.py:112
↓ 12 callersMethodgenerate_answer
( self, pixel_values: torch.Tensor, questions: List[str], return_string_probabilities: Optional[List[s
vlm_eval/models/llava.py:358
↓ 9 callersMethoddecode
(self, output_ids: Union[torch.Tensor, Sequence[int]], **kwargs)
vlm_eval/util/interfaces.py:26
↓ 8 callersMethodencode
(self, inputs: str, add_special_tokens: bool = False)
vlm_eval/util/interfaces.py:23
↓ 8 callersMethodload
(self)
vlm_eval/util/interfaces.py:42
↓ 4 callersMethodgenerate
( self, image: Image, input_text: str, do_sample: bool = True, tempera
vlm_eval/models/llava.py:411
↓ 4 callersFunctionlength
(r)
vlm_eval/util/evaluation/gqa/eval.py:306
↓ 3 callersMethodevaluate
(self)
vlm_eval/util/evaluation/vizwiz/eval.py:192
↓ 3 callersMethodgetRefIds
(self, image_ids=None, cat_ids=None, ref_ids=None, split="")
vlm_eval/util/loading/refer.py:156
↓ 3 callersMethodget_vqa_chat_prompt_fn
Generates the full reference prompt for VQA tasks.
vlm_eval/models/llava.py:228
↓ 3 callersMethodget_vqa_chat_prompt_fn
Generates the full reference prompt for VQA tasks.
vlm_eval/models/instructblip.py:123
↓ 3 callersMethodget_vqa_chat_prompt_fn
Generates the full reference prompt for VQA tasks.
vlm_eval/models/prismatic.py:117
↓ 3 callersMethodget_worker_status
(self, worker_name: str)
vlm_eval/serve/controller.py:90
↓ 3 callersFunctionload_vlm
( model_family: str, model_id: str, run_dir: Path, hf_token: Optional[str] = None, ocr: Op
vlm_eval/models/__init__.py:14
↓ 3 callersFunctionxrange
(c)
vlm_eval/util/evaluation/gqa/eval.py:302
↓ 3 callersFunctionyrange
(c)
vlm_eval/util/evaluation/gqa/eval.py:298
↓ 2 callersFunctionavg
(l)
vlm_eval/util/evaluation/gqa/eval.py:181
↓ 2 callersMethodconvert_results
TextVQA Evaluation Script expects List[{"pred_answer": str, "gt_answers": List[str] (lower, no punkt)}].
vlm_eval/tasks/harnesses/textvqa.py:288
↓ 2 callersFunctioncounts2vec
r"""Function maps counts of ngram to vector of tfidf weights.
vlm_eval/util/evaluation/nocaps/metrics.py:29
↓ 2 callersMethodcreate_index
Build index datastructures from loaded question and annotation files.
vlm_eval/util/evaluation/vqav2/eval.py:260
↓ 2 callersFunctiondownload_with_progress
Utility function for downloading files from the internet, with a handy Rich-based progress bar.
vlm_eval/tasks/download.py:24
↓ 2 callersMethodeval_pred_list
(self, pred_list)
vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:285
↓ 2 callersFunctionevaluate_after_parse
(cfg, vlm=None)
scripts/evaluate.py:77
↓ 2 callersFunctionget_conv_log_filename
()
vlm_eval/serve/gradio_web_server.py:35
↓ 2 callersMethodget_mc_prompt_fn
Generates the full reference prompt for a multiple-choice question-answer task.
vlm_eval/models/llava.py:305
↓ 2 callersMethodget_mc_prompt_fn
Generates the full reference prompt for a multiple choice question-answering task.
vlm_eval/models/prismatic.py:176
↓ 2 callersFunctionget_model_list
()
vlm_eval/serve/gradio_web_server.py:41
↓ 2 callersMethodget_prompt_fn
(self, dataset_family: str = "vqa-v2")
vlm_eval/models/llava.py:174
↓ 2 callersMethodget_question_ids
Get the list of unique question IDs that satisfy given filter conditions; default behavior returns all IDs. :param image_ids: List o
vlm_eval/util/evaluation/vqav2/eval.py:268
↓ 2 callersMethodget_queue_length
(self)
scripts/interactive_demo.py:85
↓ 2 callersMethodget_status
(self)
scripts/interactive_demo.py:95
↓ 2 callersMethodget_worker_address
(self, model_name: str)
vlm_eval/serve/controller.py:171
↓ 2 callersFunctionintersection
(r1, r2)
vlm_eval/util/evaluation/gqa/eval.py:316
↓ 2 callersFunctionis_metric
(key)
scripts/compute_z_scores.py:26
↓ 2 callersFunctionloadFile
(name)
vlm_eval/util/evaluation/gqa/eval.py:123
↓ 2 callersMethodprocessDigitArticle
(self, inText)
vlm_eval/util/evaluation/vizwiz/eval.py:264
↓ 2 callersMethodprocessPunctuation
(self, inText)
vlm_eval/util/evaluation/vizwiz/eval.py:254
↓ 2 callersFunctionprocess_punctuation
(text: str)
vlm_eval/util/evaluation/vqav2/eval.py:214
↓ 2 callersMethodregister_to_controller
(self)
scripts/interactive_demo.py:62
↓ 2 callersMethodregister_worker
(self, worker_name: str, check_heart_beat: bool, worker_status: dict)
vlm_eval/serve/controller.py:68
↓ 2 callersMethodremove_worker
(self, worker_name: str)
vlm_eval/serve/controller.py:103
↓ 2 callersFunctionscore_after_parse
(cfg)
scripts/score.py:59
↓ 2 callersMethodsend_heart_beat
(self)
scripts/interactive_demo.py:68
↓ 2 callersMethodset_generate_kwargs
(self, generate_kwargs)
vlm_eval/models/llava.py:171
↓ 2 callersFunctionto_ngrams
r"""Convert a sentence into n-grams and their counts.
vlm_eval/util/evaluation/nocaps/metrics.py:18
↓ 1 callersMethod_compute_answer_scores
compute the accuracy (soft score) of human answers
vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:223
↓ 1 callersFunctionbox_xywh2xyxy
(bbox_xywh: List[float], img_wh: Tuple[int, int], do_normalize: bool = True)
vlm_eval/tasks/harnesses/refcoco.py:33
↓ 1 callersFunctionbox_xyxy2normalized
(bbox_xyxy: List[int], img_wh: Tuple[int, int])
vlm_eval/tasks/harnesses/ocidref.py:46
↓ 1 callersFunctionbuild_demo
(embed_mode)
vlm_eval/serve/gradio_web_server.py:281
↓ 1 callersFunctionbuild_index_datasets
Given a dataset identifier and optional list of dataset sizes, return a set of PyTorch Map-style Datasets (building metadata/index files if n
vlm_eval/tasks/builders.py:47
↓ 1 callersFunctionchiSquare
(goldDist, predictedDist)
vlm_eval/util/evaluation/gqa/eval.py:380
↓ 1 callersFunctioncompute_iou
Computes IOU between two bboxes in xyxy format.
vlm_eval/tasks/harnesses/refcoco.py:305
↓ 1 callersFunctioncompute_iou
Computes IOU between two bboxes in xyxy format.
vlm_eval/tasks/harnesses/ocidref.py:303
↓ 1 callersMethodconvert_results
GQA Evaluation Script expects List[{"questionId": str, "prediction": str (lower case, no punkt)}].
vlm_eval/tasks/harnesses/gqa.py:271
↓ 1 callersMethodconvert_results
VQAv2 Evaluation Script expects List[{"question_id": int, "answer": str}] (normalizes automatically).
vlm_eval/tasks/harnesses/vqav2.py:290
↓ 1 callersMethodconvert_results
(self)
vlm_eval/tasks/harnesses/pope.py:296
↓ 1 callersMethodconvert_results
(self)
vlm_eval/tasks/harnesses/vizwiz.py:292
↓ 1 callersMethodcreateIndex
Create mappings: 1) Refs: {ref_id: ref} 2) Anns: {ann_id: ann} 3) Imgs: {image_i
vlm_eval/util/loading/refer.py:86
↓ 1 callersFunctiondownload_extract
Download all files for a given dataset (querying registry above), extracting archives if necessary.
vlm_eval/tasks/download.py:119
↓ 1 callersFunctionevaluate
(cfg: EvaluationConfig)
scripts/evaluate.py:111
↓ 1 callersMethodevaluate
Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.
vlm_eval/tasks/harnesses/gqa.py:204
↓ 1 callersMethodevaluate
Core Evaluation --> iterates through (specified) questions, computing accuracies per split.
vlm_eval/util/evaluation/vqav2/eval.py:420
↓ 1 callersMethodevaluate_unanswerability
(self)
vlm_eval/util/evaluation/vizwiz/eval.py:236
↓ 1 callersFunctionextract_with_progress
Utility function for extracting compressed archives, with a handy Rich-based progress bar.
vlm_eval/tasks/download.py:67
↓ 1 callersMethodfinalize_accuracies
Create readable accuracy dictionary with overall/split metrics.
vlm_eval/util/evaluation/vqav2/eval.py:458
↓ 1 callersMethodfrom_str
(cls, name)
vlm_eval/serve/controller.py:33
↓ 1 callersMethodgenerate_stream
(self, params)
scripts/interactive_demo.py:103
↓ 1 callersMethodgenerate_stream_gate
(self, params)
scripts/interactive_demo.py:150
↓ 1 callersMethodgetMask
(self, ref: str)
vlm_eval/util/loading/refer.py:331
↓ 1 callersMethodgetRefBox
(self, ref_id)
vlm_eval/util/loading/refer.py:276
↓ 1 callersFunctiongetStepsNum
(question)
vlm_eval/util/evaluation/gqa/eval.py:226
↓ 1 callersFunctiongetWordsNum
(question)
vlm_eval/util/evaluation/gqa/eval.py:221
↓ 1 callersMethodget_anls
(self, s1, s2)
vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:278
↓ 1 callersMethodget_answerable_preds
(self)
vlm_eval/util/evaluation/vizwiz/eval.py:225
↓ 1 callersMethodget_bbox_refer_chat_prompt_fn
Generates the full reference prompt for a referring expression localization task.
vlm_eval/models/llava.py:333
↓ 1 callersMethodget_bbox_refer_chat_prompt_fn
Generates the full reference prompt for a referring expression localization task.
vlm_eval/models/instructblip.py:158
↓ 1 callersMethodget_bbox_refer_chat_prompt_fn
Generates the full reference prompt for a referring expression localization task.
vlm_eval/models/prismatic.py:199
↓ 1 callersMethodget_captioning_prompt_fn
Generates the full reference prompt for captioning tasks.
vlm_eval/models/llava.py:203
↓ 1 callersMethodget_captioning_prompt_fn
Generates the full reference prompt for captioning tasks.
vlm_eval/models/instructblip.py:114
↓ 1 callersMethodget_captioning_prompt_fn
Generates the full reference prompt for captioning tasks.
vlm_eval/models/prismatic.py:101
↓ 1 callersMethodget_contrast_caption_chat_prompt_fn
Generates the full reference prompt for a multi-pair contrast captioning task (e.g., WinoGround).
vlm_eval/models/llava.py:283
↓ 1 callersMethodget_contrast_caption_chat_prompt_fn
Generates the full reference prompt for a multi-pair contrast captioning task (e.g., WinoGround).
vlm_eval/models/instructblip.py:149
↓ 1 callersMethodget_contrast_caption_chat_prompt_fn
Generates the full reference prompt for a multi-pair contrast captioning task (e.g., WinoGround).
vlm_eval/models/prismatic.py:159
↓ 1 callersFunctionget_runid
(filepath)
scripts/compute_z_scores.py:30
↓ 1 callersFunctionget_scorer
( dataset_family: str, dataset_id: str, task_results_dir: Path, full_results: Dict[str, Dict],
vlm_eval/tasks/harnesses/__init__.py:71
↓ 1 callersFunctionget_split
(take_id: int)
vlm_eval/tasks/harnesses/ocidref.py:40
↓ 1 callersFunctionget_task_runner
( dataset_family: str, root_dir: Path, index_file: Path, task_results_dir: Path, model_id:
vlm_eval/tasks/harnesses/__init__.py:58
↓ 1 callersMethodget_true_false_chat_prompt_fn
Generates the full reference prompt for a True/False captioning task.
vlm_eval/models/llava.py:261
↓ 1 callersMethodget_true_false_chat_prompt_fn
Generates the full reference prompt for a True/False captioning task.
vlm_eval/models/instructblip.py:140
↓ 1 callersMethodget_true_false_chat_prompt_fn
Generates the full reference prompt for a True/False captioning task.
vlm_eval/models/prismatic.py:143
↓ 1 callersMethodget_worker_address_lottery
(self, model_name: str)
vlm_eval/serve/controller.py:122
↓ 1 callersMethodget_worker_address_shortest_queue
(self, model_name: str)
vlm_eval/serve/controller.py:156
↓ 1 callersFunctioninteractive_demo
(cfg: DemoConfig)
scripts/interactive_demo.py:253
↓ 1 callersFunctionintersectionSize
(c1, c2)
vlm_eval/util/evaluation/gqa/eval.py:323
↓ 1 callersMethodlist_models
(self)
vlm_eval/serve/controller.py:114
↓ 1 callersMethodload
Loads model and processors (InstructBLIPProcessor contains Vicuna Tokenizer, Q-Former Tokenizer, and an ImageProcessor) using the HF
vlm_eval/models/instructblip.py:67
↓ 1 callersMethodload
Load a Prismatic/Quartz Model using the default `prisma.load_pretrained_vlm` initializer.
vlm_eval/models/prismatic.py:48
↓ 1 callersMethodloadRefs
(self, ref_ids=None)
vlm_eval/util/loading/refer.py:243
↓ 1 callersMethodload_result_file
Create a separate VQAIndex populated from a file with predicted answers (results).
vlm_eval/util/evaluation/vqav2/eval.py:357
next →1–100 of 337, ranked by callers