Functions — github.com/TRI-ML/vlm-evaluation

hub / github.com/TRI-ML/vlm-evaluation / functions

Functions337 in github.com/TRI-ML/vlm-evaluation

⨍ Functions 337◇ Types & classes 99 ↳ Endpoints 9

Functionload_demo

(url_params, request: gr.Request)

vlm_eval/serve/gradio_web_server.py:63

Functionload_demo_refresh_model_list

(request: gr.Request)

vlm_eval/serve/gradio_web_server.py:76

Methodmc_prompt_fn

(question: str, choices: List[str])

vlm_eval/models/prismatic.py:180

Methodprocess

(self, msg: str, kwargs: MutableMapping[str, Any])

vlm_eval/overwatch/overwatch.py:40

Functionprocess_question

(question: str, max_words: int = 128)

vlm_eval/util/preprocessing.py:11

Methodrank

(self)

vlm_eval/overwatch/overwatch.py:70

Methodrank

()

vlm_eval/overwatch/overwatch.py:104

Methodrank_zero_only

(self)

vlm_eval/overwatch/overwatch.py:64

Methodrank_zero_only

()

vlm_eval/overwatch/overwatch.py:93

Functionreceive_heart_beat

(request: Request)

vlm_eval/serve/controller.py:271

Functionrefresh_all_workers

()

vlm_eval/serve/controller.py:253

Functionregenerate

(state, image_process_mode, request: gr.Request)

vlm_eval/serve/gradio_web_server.py:96

Functionregister_worker

(request: Request)

vlm_eval/serve/controller.py:247

Functionrelease_model_semaphore

(fn=None)

scripts/interactive_demo.py:180

Methodscore

Run exact-match based scoring on True/False outputs, as well as ROC-AUC and PR-AUC computation.

vlm_eval/tasks/harnesses/vsr.py:254

Methodscore

Run exact-match based scoring on multiple choice outputs, as well as ROC-AUC and PR-AUC computation.

vlm_eval/tasks/harnesses/ai2d.py:293

Methodscore

Run Acc @ 0.5 IOU scoring on the predicted normalized boxes [x1 y1 x2 y2]; invalid outputs are failures.

vlm_eval/tasks/harnesses/refcoco.py:343

Methodscore

(self, model_id: str)

vlm_eval/tasks/harnesses/__init__.py:25

Methodscore

Run exact-match based scoring on multiple choice outputs, as well as ROC-AUC and PR-AUC computation.

vlm_eval/tasks/harnesses/tallyqa.py:277

Methodscore

Call wrapped functions in `vlm_eval.util.evaluation.textvqa.eval`; returns accuracy/metrics.

vlm_eval/tasks/harnesses/textvqa.py:313

Methodscore

Call wrapped functions in `vlm_eval.util.evaluation.vqav2.eval`; returns accuracy/metrics.

vlm_eval/tasks/harnesses/vqav2.py:306

Methodscore

Evaluate Yes/No VQA Results for each POPE split, returning aggregate/split accuracies.

vlm_eval/tasks/harnesses/pope.py:306

Methodscore

(self, model_id: str)

vlm_eval/tasks/harnesses/vizwiz.py:311

Methodscore

Run Acc @ 0.25 IOU scoring on the predicted normalized boxes [x1 y1 x2 y2]; invalid outputs are failures.

vlm_eval/tasks/harnesses/ocidref.py:341

Methodset_generate_kwargs

(self, generate_kwargs)

vlm_eval/models/instructblip.py:85

Methodset_generate_kwargs

(self, generate_kwargs)

vlm_eval/models/prismatic.py:66

MethodshowMask

(self, ref: str)

vlm_eval/util/loading/refer.py:351

Methodtrue_false_prompt_fn

(caption: str)

vlm_eval/models/instructblip.py:143

Methodtrue_false_prompt_fn

(caption: str)

vlm_eval/models/prismatic.py:147

Functionvote_last_response

(state, vote_type, model_selector, request: gr.Request)

vlm_eval/serve/gradio_web_server.py:84

Methodvqa_prompt_fn

(question: str)

vlm_eval/models/instructblip.py:126

Methodvqa_prompt_fn

(question: str)

vlm_eval/models/prismatic.py:121

Functionwavg

(l, w)

vlm_eval/util/evaluation/gqa/eval.py:187

Functionworker_api_generate_stream

(request: Request)

vlm_eval/serve/controller.py:278

Functionworker_api_get_status

(request: Request)

vlm_eval/serve/controller.py:285

Methodworld_size

(self)

vlm_eval/overwatch/overwatch.py:73

Methodworld_size

()

vlm_eval/overwatch/overwatch.py:108

← previous301–337 of 337, ranked by callers