Functions — github.com/JIA-Lab-research/LISA

CodeHub MCPcopy Create free account

hub / github.com/JIA-Lab-research/LISA / functions

Functions403 in github.com/JIA-Lab-research/LISA

⨍ Functions 403◇ Types & classes 70

model/segment_anything/predictor.py:274

model/segment_anything/modeling/sam.py:52

utils/conversation.py:136

model/llava/conversation.py:217

Functiondisable_torch_init

Disable the redundant torch default initialization to accelerate model creation.

model/llava/utils.py:98

Methoddisplay_summary

utils/utils.py:142

model/llava/model/multimodal_encoder/clip_encoder.py:67

Methoddummy_feature

model/llava/model/multimodal_encoder/clip_encoder.py:63

Methodexpand2square

(pil_img, background_color=(122, 116, 104))

model/llava/conversation.py:125

Methodexpand2square

(pil_img, background_color)

model/llava/train/train.py:713

Functionflash_attn_fn

( query, key, value, n_heads, past_key_value=None, softmax_scale=None, attn_bias=N

model/llava/model/language_model/mpt/attention.py:115

model/llava/utils.py:92

Functionforward

Wraps original forward to enable PrefixLM attention.

model/llava/model/language_model/mpt/hf_prefixlm_converter.py:90

Functionforward

Input shape: Batch x Time x Channel attention_mask: [bsz, q_len]

model/llava/train/llama_flash_attn_monkey_patch.py:21

Predict masks given image and prompt embeddings. Arguments: image_embeddings (torch.Tensor): the embeddings from the image

model/segment_anything/modeling/mask_decoder.py:75

model/segment_anything/modeling/mask_decoder.py:186

(self, x: torch.Tensor)

model/segment_anything/modeling/image_encoder.py:110

(self, x: torch.Tensor)

model/segment_anything/modeling/image_encoder.py:177

(self, x: torch.Tensor)

model/segment_anything/modeling/image_encoder.py:235

(self, x: torch.Tensor)

model/segment_anything/modeling/image_encoder.py:422

(self, x: torch.Tensor)

model/segment_anything/modeling/common.py:25

(self, x: torch.Tensor)

model/segment_anything/modeling/common.py:38

Args: image_embedding (torch.Tensor): image to attend to. Should be shape B x embedding_dim x h x w for any h and w.

model/segment_anything/modeling/transformer.py:62

( self, queries: Tensor, keys: Tensor, query_pe: Tensor, key_pe: Tensor )

model/segment_anything/modeling/transformer.py:151

(self, q: Tensor, k: Tensor, v: Tensor)

model/segment_anything/modeling/transformer.py:220

Predicts masks end-to-end from provided images and prompts. If prompts are not known in advance, using SamPredictor is recomm

model/segment_anything/modeling/sam.py:56

Embeds different types of prompts, returning both sparse and dense embeddings. Arguments: points (tuple(torch.Tens

model/segment_anything/modeling/prompt_encoder.py:140

Generate positional encoding for a grid of the specified size.

model/segment_anything/modeling/prompt_encoder.py:216

( self, image_embeddings: torch.Tensor, point_coords: torch.Tensor, point_labe

model/segment_anything/utils/onnx.py:121

( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = N

model/llava/model/language_model/llava_llama.py:55

( self, input_ids: torch.LongTensor, past_key_values: Optional[List[Tuple[torch.FloatT

model/llava/model/language_model/llava_mpt.py:73

qkv: (batch, seqlen, 3, nheads, headdim) bias: optional, shape broadcastible to (batch, nheads, seqlen, seqlen). For exam

model/llava/model/language_model/mpt/flash_attn_triton.py:940

q: (batch, seqlen_q, nheads, headdim) kv: (batch, seqlen_k, 2, nheads, headdim) bias: optional, shape broadcastible to (batch

model/llava/model/language_model/mpt/flash_attn_triton.py:991

q: (batch_size, seqlen_q, nheads, headdim) k, v: (batch_size, seqlen_k, nheads, headdim) bias: optional, shape broadcastible

model/llava/model/language_model/mpt/flash_attn_triton.py:1044

model/llava/model/language_model/mpt/blocks.py:21

( self, x: torch.Tensor, past_key_value: Optional[Tuple[torch.Tensor]] = None,

model/llava/model/language_model/mpt/blocks.py:72

( self, input_ids: torch.LongTensor, past_key_values: Optional[List[Tuple[torch.FloatT

model/llava/model/language_model/mpt/modeling_mpt.py:208

( self, input_ids: torch.LongTensor, past_key_values: Optional[List[Tuple[torch.FloatT

model/llava/model/language_model/mpt/modeling_mpt.py:419

model/llava/model/language_model/mpt/norm.py:33

model/llava/model/language_model/mpt/norm.py:74

model/llava/model/language_model/mpt/norm.py:90

( self, x, past_key_value=None, attn_bias=None, attention_mask=None,

model/llava/model/language_model/mpt/attention.py:322

( self, x, past_key_value=None, attn_bias=None, attention_mask=None,

model/llava/model/language_model/mpt/attention.py:419

(self, input: Tensor, unembed: bool = False)

model/llava/model/language_model/mpt/custom_embedding.py:8

model/llava/model/multimodal_encoder/clip_encoder.py:42

Methodfsdp_wrap_fn

model/llava/model/language_model/mpt/modeling_mpt.py:370

Methodfsdp_wrap_fn

model/llava/model/language_model/mpt/modeling_mpt.py:485

Functiongenerate

Wraps original generate to enable PrefixLM attention.

model/llava/model/language_model/mpt/hf_prefixlm_converter.py:167

MethodgetCatIds

utils/grefer.py:219

MethodgetMaskByRef

(self, ref=None, ref_id=None, merge=False)

utils/grefer.py:322

Methodget_decoder

model/llava/model/language_model/mpt/modeling_mpt.py:416

Functionget_default_conv_template

utils/conversation.py:283

Methodget_image_embedding

Returns the image embeddings for the currently set image, with shape 1xCxHxW, where C is the embedding dimension and (H,W) are

model/segment_anything/predictor.py:258

Methodget_input_embeddings

model/llava/model/language_model/mpt/modeling_mpt.py:401

Methodget_model

model/llava/model/language_model/llava_llama.py:52

Methodget_model

model/llava/model/language_model/llava_mpt.py:66

Functionget_model_name_from_path

model/llava/mm_utils.py:47

(weights, keyword)

model/llava/model/llava_arch.py:73

Methodhidden_size

model/llava/model/multimodal_encoder/clip_encoder.py:82

Functioninference

(input_str, input_image)

Functioninit_ade20k

(base_image_dir)

utils/sem_seg_dataset.py:39

Functioninit_cocostuff

(base_image_dir)

utils/sem_seg_dataset.py:69

Functioninit_empty_weights

Meta initialization context manager. A context manager under which models are initialized with all parameters on the meta device, therefore c

model/llava/model/language_model/mpt/meta_init_context.py:8

Functioninit_mapillary

(base_image_dir)

utils/sem_seg_dataset.py:20

Functioninit_paco_lvis

(base_image_dir)

utils/sem_seg_dataset.py:88

Functioninit_pascal_part

(base_image_dir)

utils/sem_seg_dataset.py:112

Functioninit_to_zero

model/llava/model/language_model/mpt/flash_attn_triton.py:574

Functionkaiming_normal_param_init_fn_

( module: nn.Module, n_layers: int, d_model: Optional[int] = None, init_div_is_residual: Union

model/llava/model/language_model/mpt/param_init_fns.py:314

Functionkaiming_uniform_param_init_fn_

( module: nn.Module, n_layers: int, d_model: Optional[int] = None, init_div_is_residual: Union

model/llava/model/language_model/mpt/param_init_fns.py:277

(self, cat_ids=[])

utils/grefer.py:233

Functionload_image_from_base64

model/llava/mm_utils.py:11

Functionload_pretrained_model

( model_path, model_base, model_name, load_8bit=False, load_4bit=False, device_map="au

model/llava/model/builder.py:27

Functionmake_inputs_require_grad

(module, input, output)

model/llava/train/train.py:891

Functionneox_param_init_fn_

From section 2.3.1 of GPT-NeoX-20B: An Open-Source AutoregressiveLanguage Model — Black et. al. (2022) see https://github.com/EleutherAI/gpt-

model/llava/model/language_model/mpt/param_init_fns.py:247

Methodnum_patches

model/llava/model/multimodal_encoder/clip_encoder.py:86

Methodparam_init_fn

model/llava/model/language_model/mpt/modeling_mpt.py:361

Methodparam_init_fn

model/llava/model/language_model/mpt/modeling_mpt.py:476

Predict masks for the given input prompts, using the currently set image. Arguments: point_coords (np.ndarray or None): A

model/segment_anything/predictor.py:93

Functionprepare_inputs_for_generation

( self: BloomForCausalLM, input_ids: torch.LongTensor, past: Optional[torch.Tensor] =

model/llava/model/language_model/mpt/hf_prefixlm_converter.py:499

Methodprepare_inputs_for_generation

( self, input_ids, past_key_values=None, attention_mask=None, inputs_e

model/llava/model/language_model/llava_llama.py:137

Methodprepare_inputs_for_generation

( self, input_ids, past_key_values=None, inputs_embeds=None, **kwargs )

model/llava/model/language_model/llava_mpt.py:138

Methodprepare_inputs_for_generation

( self, input_ids, past_key_values=None, inputs_embeds=None, **kwargs )

model/llava/model/language_model/mpt/modeling_mpt.py:491

Functionpretty_print_semaphore

model/llava/utils.py:131

Functionprocess_images

(images, image_processor, model_cfg)

model/llava/mm_utils.py:15

Functionregister_empty_buffer

(module, name, buffer)

model/llava/model/language_model/mpt/meta_init_context.py:72

Functionregister_empty_parameter

(module, name, param)

model/llava/model/language_model/mpt/meta_init_context.py:63

Functionscaled_multihead_dot_product_attention

( query, key, value, n_heads, past_key_value=None, softmax_scale=None, attn_bias=N

model/llava/model/language_model/mpt/attention.py:28

Methodset_decoder

(self, decoder)

model/llava/model/language_model/mpt/modeling_mpt.py:413

Methodset_input_embeddings

model/llava/model/language_model/mpt/modeling_mpt.py:112

Methodset_input_embeddings

model/llava/model/language_model/mpt/modeling_mpt.py:404

Methodset_output_embeddings

(self, new_embeddings)

model/llava/model/language_model/mpt/modeling_mpt.py:410

utils/grefer.py:348

utils/refer.py:360

(self, ref, seg_box="seg")

utils/grefer.py:240

Methodto_gradio_chatbot

utils/conversation.py:112

Methodto_gradio_chatbot

model/llava/conversation.py:171

Functiontorch_default_param_init_fn_

(module: nn.Module, verbose: int = 0, **kwargs)

model/llava/model/language_model/mpt/param_init_fns.py:13

Functiontriton_flash_attn_fn

( query, key, value, n_heads, past_key_value=None, softmax_scale=None, attn_bias=N

model/llava/model/language_model/mpt/attention.py:190

Functionviolates_moderation

Check whether the text violates OpenAI moderation API.

model/llava/utils.py:108

Functionwrapper

(*args, **kwargs)

model/llava/model/language_model/mpt/meta_init_context.py:86

← previous next →301–400 of 403, ranked by callers