MCPcopy Create free account

hub / github.com/JIA-Lab-research/LISA / functions

Functions403 in github.com/JIA-Lab-research/LISA

⨍ Functions 403◇ Types & classes 70
↓ 54 callersMethodfrom_pretrained
See `AutoTokenizer.from_pretrained` docstring.
model/llava/model/language_model/mpt/adapt_tokenizer.py:42
↓ 39 callersMethodget_model
(self)
model/llava/model/llava_arch.py:87
↓ 35 callersMethodcat
(self, new_stats: "MaskData")
model/segment_anything/utils/amg.py:59
↓ 21 callersMethoditems
(self)
model/segment_anything/utils/amg.py:41
↓ 19 callersMethodappend_message
(self, role, message)
utils/conversation.py:109
↓ 19 callersFunctiontokenizer_image_token
( prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None )
model/llava/mm_utils.py:19
↓ 16 callersMethodupdate
(self, val, n=1)
utils/utils.py:67
↓ 14 callersMethodcopy
(self)
utils/conversation.py:121
↓ 12 callersMethod_toList
(x)
utils/grefer.py:75
↓ 12 callersMethodget_vision_tower
(self)
model/llava/model/llava_arch.py:37
↓ 10 callersMethodall_reduce
(self)
utils/utils.py:73
↓ 10 callersMethodembed_tokens
(self, x)
model/llava/model/language_model/llava_mpt.py:40
↓ 9 callersMethodapply_image
Expects a numpy array with shape HxWxC in uint8 format.
model/segment_anything/utils/transforms.py:27
↓ 8 callersMethodget_prompt
(self)
utils/conversation.py:48
↓ 8 callersMethodreset
(self)
utils/utils.py:61
↓ 6 callersMethodfilter
(self, keep: torch.Tensor)
model/segment_anything/utils/amg.py:44
↓ 5 callersFunction_cast_if_autocast_enabled
(tensor)
model/llava/model/language_model/mpt/norm.py:4
↓ 5 callersMethodcopy
(self)
model/llava/conversation.py:205
↓ 5 callersFunctiongeneric_param_init_fn_
( module: nn.Module, init_fn_, n_layers: int, d_model: Optional[int] = None, init_div_is_r
model/llava/model/language_model/mpt/param_init_fns.py:33
↓ 5 callersMethodget_dense_pe
Returns the positional encoding used to encode point prompts, applied to a dense set of points the shape of the image encoding.
model/segment_anything/modeling/prompt_encoder.py:67
↓ 5 callersMethodinitialize_vision_modules
(self, model_args, fsdp=None)
model/llava/model/llava_arch.py:43
↓ 4 callersFunctioncall_og_forward
()
model/llava/model/language_model/mpt/hf_prefixlm_converter.py:108
↓ 4 callersMethodgetImgIds
(self, ref_ids=[])
utils/refer.py:205
↓ 4 callersMethodgetRefIds
(self, image_ids=[], cat_ids=[], ref_ids=[], split="")
utils/refer.py:144
↓ 4 callersMethodget_input_embeddings
(self)
model/llava/model/language_model/mpt/modeling_mpt.py:109
↓ 4 callersMethodget_preprocess_shape
Compute the output size given input size and target long side length.
model/segment_anything/utils/transforms.py:103
↓ 4 callersMethodget_prompt
(self)
model/llava/conversation.py:31
↓ 4 callersMethodpostprocess_masks
Remove padding and upscale masks to the original image size. Arguments: masks (torch.Tensor): Batched masks from the mask_
model/segment_anything/modeling/sam.py:137
↓ 4 callersMethodpreprocess
Normalize pixel values and pad to a square input.
utils/dataset.py:341
↓ 4 callersMethodpreprocess
Normalize pixel values and pad to a square input.
model/segment_anything/modeling/sam.py:174
↓ 3 callersMethod__init__
Args: dim (int): Number of input channels. num_heads (int): Number of attention heads in each ViT block.
model/segment_anything/modeling/image_encoder.py:131
↓ 3 callersFunction_build_sam
( encoder_embed_dim, encoder_depth, encoder_num_heads, encoder_global_attn_indexes, checkp
model/segment_anything/build_sam.py:56
↓ 3 callersFunction_flash_attn_backward
( do, q, k, v, o, lse, dq, dk, dv, bias=None, causal=False, softmax_scale=None )
model/llava/model/language_model/mpt/flash_attn_triton.py:833
↓ 3 callersFunction_flash_attn_forward
(q, k, v, bias=None, causal=False, softmax_scale=None)
model/llava/model/language_model/mpt/flash_attn_triton.py:751
↓ 3 callersMethod_pe_encoding
Positionally encode points that are normalized to [0,1].
model/segment_anything/modeling/prompt_encoder.py:203
↓ 3 callersMethod_separate_heads
(self, x: Tensor, num_heads: int)
model/segment_anything/modeling/transformer.py:210
↓ 3 callersMethodappend_message
(self, role, message)
model/llava/conversation.py:109
↓ 3 callersMethodapply_coords
Expects a numpy array of length 2 in the final dimension. Requires the original image size in (H, W) format.
model/segment_anything/utils/transforms.py:36
↓ 3 callersFunctioncalculate_stability_score
Computes the stability score for a batch of masks. The stability score is the IoU between the binary masks obtained by thresholding the p
model/segment_anything/utils/amg.py:156
↓ 3 callersFunctionget_mask_from_json
(json_path, img)
utils/data_processing.py:9
↓ 3 callersMethodget_output_embeddings
(self)
model/llava/model/language_model/mpt/modeling_mpt.py:407
↓ 3 callersMethodinitialize_lisa_modules
(self, config)
model/LISA.py:79
↓ 3 callersMethodloadImgs
(self, image_ids=[])
utils/refer.py:229
↓ 3 callersMethodloadRefs
(self, ref_ids=[])
utils/refer.py:217
↓ 3 callersFunctionmaybe_zero_3
(param, ignore_status=False, name=None)
model/llava/train/train.py:107
↓ 3 callersMethodpreprocess
Normalize pixel values and pad to a square input.
utils/sem_seg_dataset.py:176
↓ 3 callersMethodreset_image
Resets the currently set image.
model/segment_anything/predictor.py:277
↓ 2 callersMethod__init__
( self, config, **kwargs, )
model/LISA.py:105
↓ 2 callersMethod__init__
( self, embedding_dim: int, num_heads: int, downsample_rate: int = 1, )
model/segment_anything/modeling/transformer.py:191
↓ 2 callersMethod__init__
( self, normalized_shape, eps=1e-05, weight=True, dtype=None, device=None )
model/llava/model/language_model/mpt/norm.py:62
↓ 2 callersFunction_bwd_kernel_one_col_block
( start_n, Q, K, V, Bias, DO, DQ, DK, DV, LSE, D, softmax_scal
model/llava/model/language_model/mpt/flash_attn_triton.py:350
↓ 2 callersFunction_bwd_store_dk_dv
( dk_ptrs, dv_ptrs, dk, dv, offs_n, offs_d, seqlen_k, headdim, EVEN_M: tl.
model/llava/model/language_model/mpt/flash_attn_triton.py:317
↓ 2 callersFunction_get_attn_modules
Helper that gets a list of the model's attention modules. Each module has a `bias` buffer used for causal masking. The Prefix LM conv
model/llava/model/language_model/mpt/hf_prefixlm_converter.py:63
↓ 2 callersFunction_normal_param_init_fn_
( module: nn.Module, std: float, n_layers: int, d_model: Optional[int] = None, init_div_is
model/llava/model/language_model/mpt/param_init_fns.py:168
↓ 2 callersFunction_reset_is_causal
( num_query_tokens: int, num_key_tokens: int, original_is_causal: bool )
model/llava/model/language_model/mpt/attention.py:15
↓ 2 callersMethod_set_config_defaults
(self, config, config_defaults)
model/llava/model/language_model/mpt/configuration_mpt.py:134
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
model/llava/train/train.py:254
↓ 2 callersMethod_validate_config
(self)
model/llava/model/language_model/mpt/configuration_mpt.py:140
↓ 2 callersFunctionauto_upgrade
(config)
model/llava/model/utils.py:4
↓ 2 callersFunctionbatched_mask_to_box
Calculates boxes in XYXY format around masks. Return [0,0,0,0] for an empty mask. For input shape C1xC2x...xHxW, the output shape is C1xC2x..
model/segment_anything/utils/amg.py:303
↓ 2 callersFunctionbox_xyxy_to_xywh
(box_xyxy: torch.Tensor)
model/segment_anything/utils/amg.py:91
↓ 2 callersFunctionbuild_vision_tower
(vision_tower_cfg, **kwargs)
model/llava/model/multimodal_encoder/builder.py:4
↓ 2 callersFunctioncheck_valid_inputs
(*tensors, valid_dtypes=[torch.float16, torch.bfloat16])
model/llava/model/language_model/mpt/attention.py:103
↓ 2 callersFunctioncrop_len
(orig_len, n_crops, overlap)
model/segment_anything/utils/amg.py:215
↓ 2 callersFunctiondict_to_cuda
(input_dict)
utils/utils.py:153
↓ 2 callersMethodencode_images
(self, images)
model/llava/model/llava_arch.py:93
↓ 2 callersMethodevaluate
( self, images_clip, images, input_ids, resize_list, original_
model/LISA.py:345
↓ 2 callersMethodfeature_select
(self, image_forward_outs)
model/llava/model/multimodal_encoder/clip_encoder.py:31
↓ 2 callersMethodforward
(self, **kwargs)
model/LISA.py:164
↓ 2 callersMethodforward_with_coords
Positionally encode points that are not normalized to [0,1].
model/segment_anything/modeling/prompt_encoder.py:231
↓ 2 callersMethodgetCatIds
(self)
utils/refer.py:214
↓ 2 callersMethodgetMask
(self, ann)
utils/grefer.py:302
↓ 2 callersFunctionget_rel_pos
Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (int): size of qu
model/segment_anything/modeling/image_encoder.py:321
↓ 2 callersMethodget_visual_embs
(self, pixel_values: torch.FloatTensor)
model/LISA.py:151
↓ 2 callersMethodloadAnns
(self, ann_ids=[])
utils/grefer.py:225
↓ 2 callersMethodloadCats
(self, cat_ids=[])
utils/refer.py:235
↓ 2 callersMethodload_model
(self)
model/llava/model/multimodal_encoder/clip_encoder.py:21
↓ 2 callersFunctionmask_to_rle_pytorch
Encodes masks to an uncompressed RLE, in the format expected by pycoco tools.
model/segment_anything/utils/amg.py:107
↓ 2 callersMethodpredict_masks
Predicts masks. See 'forward' for more details.
model/segment_anything/modeling/mask_decoder.py:116
↓ 2 callersMethodpredict_torch
Predict masks for the given input prompts, using the currently set image. Input prompts are batched torch tensors and are expected to
model/segment_anything/predictor.py:178
↓ 2 callersMethodprepare_inputs_labels_for_multimodal
( self, input_ids, attention_mask, past_key_values, labels, images )
model/llava/model/llava_arch.py:98
↓ 2 callersMethodpreprocess
Normalize pixel values and pad to a square input.
utils/reason_seg_dataset.py:97
↓ 2 callersMethodpreprocess
Normalize pixel values and pad to a square input.
utils/refer_seg_dataset.py:108
↓ 2 callersMethodpreprocess
Normalize pixel values and pad to a square input.
utils/vqa_dataset.py:71
↓ 2 callersFunctionrank0_print
(*args)
model/llava/train/train.py:40
↓ 2 callersFunctionremove_small_regions
Removes small disconnected regions and holes in a mask. Returns the mask and an indicator of if the mask has been modified.
model/segment_anything/utils/amg.py:267
↓ 2 callersFunctionrle_to_mask
Compute a binary mask from an uncompressed RLE.
model/segment_anything/utils/amg.py:138
↓ 2 callersFunctionrms_norm
(x, weight=None, eps=1e-05)
model/llava/model/language_model/mpt/norm.py:54
↓ 2 callersFunctiontrain
()
model/llava/train/train.py:810
↓ 2 callersFunctionuncrop_boxes_xyxy
(boxes: torch.Tensor, crop_box: List[int])
model/segment_anything/utils/amg.py:237
↓ 2 callersFunctionvalidate
(val_loader, model_engine, epoch, writer, args)
train_ds.py:523
↓ 1 callersMethod__init__
( self, input_dim: int, hidden_dim: int, output_dim: int, num_layers:
model/segment_anything/modeling/mask_decoder.py:170
↓ 1 callersMethod__init__
( self, embedding_dim: int, mlp_dim: int, act: Type[nn.Module] = nn.GELU,
model/segment_anything/modeling/common.py:14
↓ 1 callersMethod__init__
Encodes prompts for input to SAM's mask decoder. Arguments: embed_dim (int): The prompts' embedding dimension im
model/segment_anything/modeling/prompt_encoder.py:17
↓ 1 callersMethod__init__
(self, config: LlamaConfig)
model/llava/model/language_model/llava_llama.py:35
↓ 1 callersMethod__init__
(self, config: MPTConfig)
model/llava/model/language_model/llava_mpt.py:36
↓ 1 callersMethod__init__
( self, d_model: int, expansion_ratio: int, device: Optional[str] = None )
model/llava/model/language_model/mpt/blocks.py:12
↓ 1 callersMethod__init__
(self, config: MPTConfig)
model/llava/model/language_model/mpt/modeling_mpt.py:42
↓ 1 callersMethod__init__
( self, d_model: int, n_heads: int, attn_impl: str = "triton", clip_qk
model/llava/model/language_model/mpt/attention.py:268
↓ 1 callersFunction_add_speaker_and_signal
Add speaker and start/end signal on each round.
model/llava/train/train.py:292
next →1–100 of 403, ranked by callers