Functions — github.com/TRI-ML/vlm-evaluation

CodeHub MCPcopy Create free account

hub / github.com/TRI-ML/vlm-evaluation / functions

Functions337 in github.com/TRI-ML/vlm-evaluation

⨍ Functions 337◇ Types & classes 99 ↳ Endpoints 9

vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:296

(self, gts, res, n=2)

vlm_eval/util/evaluation/vizwiz/eval.py:18

VQA helper class for reading, indexing, and visualizing questions and answers. :param annotation_file: Path to VQAv2-formatted annot

vlm_eval/util/evaluation/vqav2/eval.py:245

Evaluation helper class for taking two pre-computed VQA Indices, and returning stratified results (accuracy, accuracy per question ty

vlm_eval/util/evaluation/vqav2/eval.py:404

(self, controller_addr, worker_addr, worker_id, no_register, vlm, model_name)

scripts/interactive_demo.py:50

vlm_eval/tasks/harnesses/gqa.py:129

vlm_eval/tasks/harnesses/gqa.py:170

vlm_eval/tasks/harnesses/vsr.py:111

vlm_eval/tasks/harnesses/vsr.py:152

vlm_eval/tasks/harnesses/ai2d.py:144

vlm_eval/tasks/harnesses/ai2d.py:185

vlm_eval/tasks/harnesses/refcoco.py:168

vlm_eval/tasks/harnesses/refcoco.py:209

vlm_eval/tasks/harnesses/tallyqa.py:128

vlm_eval/tasks/harnesses/tallyqa.py:169

vlm_eval/tasks/harnesses/textvqa.py:137

vlm_eval/tasks/harnesses/textvqa.py:180

vlm_eval/tasks/harnesses/vqav2.py:147

vlm_eval/tasks/harnesses/vqav2.py:189

vlm_eval/tasks/harnesses/pope.py:140

vlm_eval/tasks/harnesses/pope.py:187

vlm_eval/tasks/harnesses/vizwiz.py:139

vlm_eval/tasks/harnesses/vizwiz.py:187

vlm_eval/tasks/harnesses/ocidref.py:165

vlm_eval/tasks/harnesses/ocidref.py:206

Method__post_init__

vlm_eval/models/llava.py:57

Method__post_init__

scripts/interactive_demo.py:244

Method__post_init__

scripts/evaluate.py:72

Method__post_init__

scripts/eval_runner.py:70

Functionadd_text

(state, text, image, image_process_mode, request: gr.Request)

vlm_eval/serve/gradio_web_server.py:112

Methodbatch_decode

(self, output_ids: Union[torch.Tensor, Sequence[Sequence[int]]], **kwargs)

vlm_eval/util/interfaces.py:29

Methodbbox_refer_prompt_fn

(sentence: str)

vlm_eval/models/instructblip.py:161

Methodbbox_refer_prompt_fn

(sentence: str)

vlm_eval/models/prismatic.py:203

Functionbelongs

(element, group, question)

vlm_eval/util/evaluation/gqa/eval.py:265

Functionbuild_ai2d_indices

Parse AI2D `test.json` --> build & write index files w/ VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/ai2d.py:35

Functionbuild_gqa_indices

Parse GQA --> build & write index files w/ necessary VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/gqa.py:33

Functionbuild_ocidref_indices

Parse OCID-Ref validation sets --> build & write index files w/ necessary keys + additional metadata.

vlm_eval/tasks/harnesses/ocidref.py:61

Functionbuild_pope_indices

Parse Pope --> build & write index files w/ necessary VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/pope.py:35

Functionbuild_refcoco_indices

Parse RefCOCO* validation sets --> build & write index files w/ necessary keys + additional metadata.

vlm_eval/tasks/harnesses/refcoco.py:53

Functionbuild_tallyqa_indices

Parse TallyQA `test.json` --> build & write index files w/ VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/tallyqa.py:34

Functionbuild_textvqa_indices

Parse Text VQA raw files --> build & write index files w/ VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/textvqa.py:34

Functionbuild_vizwiz_indices

Parse VizWiz --> build & write index files w/ necessary VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/vizwiz.py:30

Functionbuild_vqav2_indices

Parse VQAv2 raw files --> build & write index files w/ necessary VQA keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/vqav2.py:35

Functionbuild_vsr_indices

Parse VSR --> build & write index files w/ necessary VSR keys + additional dataset-specific data.

vlm_eval/tasks/harnesses/vsr.py:32

Methodcaptioning_prompt_fn

vlm_eval/models/instructblip.py:117

Methodcaptioning_prompt_fn

vlm_eval/models/prismatic.py:105

r"""Compute CIDEr score given ground truth captions and predictions.

vlm_eval/util/evaluation/nocaps/metrics.py:63

Functionclear_history

(request: gr.Request)

vlm_eval/serve/gradio_web_server.py:106

Methodcontrast_caption_prompt_fn

vlm_eval/models/instructblip.py:152

Methodcontrast_caption_prompt_fn

vlm_eval/models/prismatic.py:163

Methoddataset_id

vlm_eval/conf/datasets.py:418

Methoddisplay_qa

Display the specified annotations (questions & answers).

vlm_eval/util/evaluation/vqav2/eval.py:350

Methodeval_pred_list

(self, pred_list)

vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:244

Methodeval_pred_list

(self, pred_list)

vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:260

Methodeval_pred_list

(self, pred_list)

vlm_eval/util/evaluation/textvqa/m4c_evaluators.py:315

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/vsr.py:186

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/ai2d.py:219

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/refcoco.py:243

(self, vlm: VLM, device_batch_size: int, num_workers: int)

vlm_eval/tasks/harnesses/__init__.py:20

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/tallyqa.py:203

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/textvqa.py:217

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/vqav2.py:223

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/pope.py:221

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/vizwiz.py:221

Initialize Dataloader & partition data across ranks, writing metrics to disk on termination.

vlm_eval/tasks/harnesses/ocidref.py:240

( self, image: Image, input_text: str, do_sample: bool = False, temper

vlm_eval/models/prismatic.py:229

( self, image: Image, input_text: str, do_sample: bool, temperature: f

vlm_eval/util/interfaces.py:59

Methodgenerate_answer

( self, pixel_values: torch.Tensor, questions: List[str], return_string_probabilities: Optional[List[s

vlm_eval/models/instructblip.py:167

Methodgenerate_answer

( self, pixel_values: torch.Tensor, question_prompts: List[str], return_string

vlm_eval/models/prismatic.py:218

Methodgenerate_answer

( self, pixel_values: torch.Tensor, question_prompts: List[str], return_string

vlm_eval/util/interfaces.py:51

Functiongenerate_stream

(request: Request)

scripts/interactive_demo.py:187

MethodgetAnnIds

(self, image_ids=None, cat_ids=None, ref_ids=None)

vlm_eval/util/loading/refer.py:199

MethodgetCatIds

vlm_eval/util/loading/refer.py:240

MethodgetImgIds

(self, ref_ids=None)

vlm_eval/util/loading/refer.py:229

Methodget_annotations

Load annotations for the specified question IDs. Default (qids = None) returns empty list. :param qids: List of question IDs to retr

vlm_eval/util/evaluation/vqav2/eval.py:340

Methodget_image_ids

Get the list of unique image IDs that satisfy given filter conditions; default behavior returns all IDs. :param question_ids: List o

vlm_eval/util/evaluation/vqav2/eval.py:304

Methodget_prompt_builder

(self, system_prompt: Optional[str] = None)

vlm_eval/models/prismatic.py:69

Methodget_prompt_builder

(self, system_prompt: Optional[str] = None)

vlm_eval/util/interfaces.py:45

Methodget_prompt_fn

(self, dataset_family: str = "vqa-v2")

vlm_eval/models/instructblip.py:88

Methodget_prompt_fn

(self, dataset_family: str = "vqa-v2")

vlm_eval/models/prismatic.py:72

Methodget_prompt_fn

(self, dataset_family: str = "vqa-v2")

vlm_eval/util/interfaces.py:48

Functionget_status

(request: Request)

scripts/interactive_demo.py:203

Functionget_worker_address

(request: Request)

vlm_eval/serve/controller.py:264

Functionheart_beat_controller

vlm_eval/serve/controller.py:51

Functionheart_beat_worker

scripts/interactive_demo.py:43

Functionhttp_bot

(state, model_selector, interaction_mode, temperature, max_new_tokens, request: gr.Request)

vlm_eval/serve/gradio_web_server.py:138

(fn: Callable[..., Any])

vlm_eval/overwatch/overwatch.py:94

FunctionintersectionRate

vlm_eval/util/evaluation/gqa/eval.py:327

Methodis_rank_zero

vlm_eval/overwatch/overwatch.py:67

Methodis_rank_zero

vlm_eval/overwatch/overwatch.py:100

Functionlist_models

vlm_eval/serve/controller.py:258

Methodllava_bbox_refer_prompt_fn

(sentence: str)

vlm_eval/models/llava.py:352

Methodllava_cap_prompt_fn

vlm_eval/models/llava.py:223

Methodllava_contrast_caption_prompt_fn

vlm_eval/models/llava.py:300

Methodllava_mc_prompt_fn

(question: str, choices: List[str])

vlm_eval/models/llava.py:325

Methodllava_true_false_prompt_fn

vlm_eval/models/llava.py:278

Methodllava_vqa_prompt_fn

(question: str)

vlm_eval/models/llava.py:256

(self, ann_ids=None)

vlm_eval/util/loading/refer.py:252

(self, cat_ids=None)

vlm_eval/util/loading/refer.py:268

(self, image_ids=None)

vlm_eval/util/loading/refer.py:260

← previous next →201–300 of 337, ranked by callers