MCPcopy Create free account

hub / github.com/SpatialVLA/SpatialVLA / functions

Functions244 in github.com/SpatialVLA/SpatialVLA

↓ 14 callersMethodfrom_pretrained
( cls, pretrained_model_name_or_path: Optional[Union[str, os.PathLike]], *model_args,
model/modeling_spatialvla.py:495
↓ 14 callersFunctioninvert_gripper_actions
(actions: tf.Tensor)
data/utils/data_utils.py:148
↓ 10 callersMethod__init__
(self, config)
model/modeling_gemma2.py:81
↓ 5 callersMethoddecode_token_ids_to_actions
decode token ids to continuous actions. action_token_id: np.ndarray, (n, 7), token ids. return: np.ndarray, (n, 7), continuous actions
model/action_tokenizer.py:46
↓ 5 callersFunctionrel2abs_gripper_actions
Converts relative gripper actions (+1 for closing, -1 for opening) to absolute actions (0 = closed; 1 = open). Assumes that the first relati
data/utils/data_utils.py:152
↓ 4 callersFunctionrepeat_kv
This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he
model/modeling_gemma2.py:157
↓ 3 callersFunctionbinarize_gripper_actions
Converts gripper actions from continuous to binary values (0 and 1). We exploit that fact that most of the time, the gripper is fully open (
data/utils/data_utils.py:113
↓ 3 callersMethodset_bins
(self, bin_policy)
model/action_tokenizer.py:172
↓ 2 callersMethod__init__
(self, in_channels=3, num_pos_feats=768, n_freqs=8, logscale=True)
model/modeling_spatialvla.py:46
↓ 2 callersFunction_freeze_params
(module)
train/spatialvla_pretrain.py:337
↓ 2 callersFunction_freeze_params
(module)
train/spatialvla_finetune.py:248
↓ 2 callersMethod_prepare_4d_causal_attention_mask_with_cache_position
Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_l
model/modeling_gemma2.py:831
↓ 2 callersMethod_update_causal_mask
( self, attention_mask, token_type_ids, past_key_values, cache_positio
model/modeling_spatialvla.py:258
↓ 2 callersFunctionallocate_threads
Allocates an integer number of threads across datasets based on weights. The final array sums to `n`, but each element is no less than 1. If
data/utils/data_utils.py:365
↓ 2 callersFunctionbuild_datasets
( data_args, output_dir, # NOTE: from training_args.output_dir vla_processor=None, )
data/dataset.py:175
↓ 2 callersFunctioneuler_to_rmat
(euler)
data/utils/droid_utils.py:13
↓ 2 callersMethodget_bin_policy
(self, gs_params=None, min_sigma=0.0)
model/action_tokenizer.py:343
↓ 2 callersMethodget_norm_meshgrid
(self, bin_policy)
model/action_tokenizer.py:372
↓ 2 callersFunctioninit_dist
(launcher, backend='nccl', **kwargs)
train/dist_utils.py:29
↓ 2 callersFunctionmake_dataset_from_rlds
This function is responsible for loading a specific RLDS dataset from storage and getting it into a standardized format. Yields a dataset of
data/rlds.py:37
↓ 2 callersFunctionrand_swap_exterior_images
Randomly swaps the two exterior images (for training with single exterior input).
data/utils/droid_utils.py:59
↓ 2 callersFunctionrelabel_bridge_actions
Relabels actions to use reached proprioceptive state; discards last timestep (no-action).
data/utils/data_utils.py:179
↓ 2 callersFunctionreplace_compute_loss
()
train/monkey_patch.py:328
↓ 2 callersFunctionreplace_train_dataloader
()
train/monkey_patch.py:218
↓ 2 callersFunctionreplace_train_sampler
()
train/monkey_patch.py:178
↓ 2 callersFunctionrotate_half
Rotates half the hidden dims of the input.
model/modeling_gemma2.py:123
↓ 2 callersMethodto_dict
(self)
model/configuration_spatialvla.py:116
↓ 1 callersFunction_find_free_port
()
train/dist_utils.py:13
↓ 1 callersFunction_init_dist_mpi
(backend, **kwargs)
train/dist_utils.py:48
↓ 1 callersFunction_init_dist_pytorch
(backend, **kwargs)
train/dist_utils.py:41
↓ 1 callersFunction_init_dist_slurm
Initialize slurm distributed training environment. If argument ``port`` is not specified, then the master port will be system environment var
train/dist_utils.py:60
↓ 1 callersFunction_is_free_port
(port)
train/dist_utils.py:22
↓ 1 callersMethod_norm
(self, x)
model/modeling_gemma2.py:66
↓ 1 callersMethod_reset_parameters
init with small weights to maintain stable training.
model/modeling_spatialvla.py:67
↓ 1 callersMethod_update_causal_mask
( self, attention_mask: torch.Tensor, input_tensor: torch.Tensor, cache_positi
model/modeling_gemma2.py:796
↓ 1 callersFunctionapply_frame_transforms
Applies common transforms that happen at a frame level. These transforms are usually more CPU-intensive, (e.g., decoding or resizing images).
data/rlds.py:364
↓ 1 callersFunctionapply_obs_transform
(fn: Callable[[Dict], Dict], frame: Dict)
data/rlds.py:396
↓ 1 callersFunctionapply_per_dataset_frame_transforms
Optionally applied *per-dataset* transforms that happen at a frame level. Args: chunk_filter_fn (callable, optional): Filter functio
data/rlds.py:350
↓ 1 callersFunctionapply_rotary_pos_emb
Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T
model/modeling_gemma2.py:130
↓ 1 callersFunctionapply_trajectory_transforms
Applies common transforms that happen at a trajectory level. Such transforms are usually some sort of "relabeling" (e.g., filtering, chunking
data/rlds.py:246
↓ 1 callersMethodbackproject_patch
Backproject depth map to 3D points in camera coordinate. Args: K: camera intrinsic matrix (b 3 3) depth: dept
model/modeling_spatialvla.py:195
↓ 1 callersMethodbatch_decode
This method forwards all its arguments to GemmaTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring o
model/processing_spatialvla.py:195
↓ 1 callersFunctionbuild_interleaved_dataset
Creates an interleaved dataset from list of dataset configs (kwargs). Returns a dataset of batched frames. Args: dataset_kwargs_list
data/rlds.py:464
↓ 1 callersMethodcartesian_to_spherical
(self, x, y, z)
model/action_tokenizer.py:93
↓ 1 callersFunctiondataset_statistics
( dataset_kwargs_list: List[Dict], train: bool, sample_weights: Optional[List[float]] = None, balance_weig
data/rlds.py:425
↓ 1 callersMethoddecode
This method forwards all its arguments to GemmaTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this
model/processing_spatialvla.py:203
↓ 1 callersMethoddecode_actions
( self, generation_outputs: torch.Tensor, unnorm_key: Optional[str] = None, )
model/processing_spatialvla.py:216
↓ 1 callersMethodfrequency_encoding
Embeds x to (x, sin(2^k x), cos(2^k x), ...) Different from the paper, "x" is also in the output See https://github.com/bmild
model/modeling_spatialvla.py:74
↓ 1 callersFunctionget_dataset_statistics
Either computes the statistics of a dataset or loads them from a cache file if this function has been called before with the same `hash_depen
data/utils/data_utils.py:244
↓ 1 callersMethodget_image_features
(self, pixel_values: torch.FloatTensor, intrinsic: torch.FloatTensor)
model/modeling_spatialvla.py:308
↓ 1 callersMethodget_input_embeddings
(self)
model/modeling_spatialvla.py:225
↓ 1 callersFunctionget_length_grouped_indices
(lengths, batch_size, world_size, generator=None, merge=True)
train/monkey_patch.py:100
↓ 1 callersMethodget_output_embeddings
(self)
model/modeling_gemma2.py:906
↓ 1 callersFunctionget_oxe_dataset_kwargs_and_weights
Generates dataset kwargs for a given dataset mix from the Open X-Embodiment dataset. The returned kwargs (per-dataset configs) and weights ca
data/oxe/__init__.py:89
↓ 1 callersFunctioninvert_rmat
(rot_mat)
data/utils/droid_utils.py:17
↓ 1 callersFunctionmain
()
train/spatialvla_pretrain.py:159
↓ 1 callersFunctionmain
()
train/spatialvla_finetune.py:147
↓ 1 callersFunctionmake_oxe_dataset_kwargs
Generates config (kwargs) for given dataset from Open-X Embodiment.
data/oxe/__init__.py:11
↓ 1 callersMethodmulti_modal_get_item
(self, data_item)
data/dataset.py:121
↓ 1 callersFunctionpprint_data_mixture
( dataset_kwargs_list: List[Dict[str, Any]], dataset_weights: List[int] )
data/utils/data_utils.py:193
↓ 1 callersMethodpredict_action
( self, model_inputs, )
model/modeling_spatialvla.py:485
↓ 1 callersFunctionprocess_zoe
https://github.com/huggingface/transformers/blob/v4.45.2/src/transformers/models/zoedepth/image_processing_zoedepth.py
model/modeling_spatialvla.py:99
↓ 1 callersMethodresize_token_embeddings
( self, new_num_tokens: Optional[int] = None, pad_to_multiple_of: Optional[int] = None
model/modeling_spatialvla.py:246
↓ 1 callersFunctionrotmat_to_rot6d
Converts rotation matrix to R6 rotation representation (first two rows in rotation matrix). Args: mat: rotation matrix Returns:
data/utils/droid_utils.py:21
↓ 1 callersFunctionsave_dataset_statistics
Saves a `ds_stats.json` file.
data/utils/data_utils.py:345
↓ 1 callersMethodset_bins
(self, bin_policy)
model/action_tokenizer.py:88
↓ 1 callersMethodspatial_embedding_adaption
gs_params0, gs_params1: Dict embeddings: tensor (S,E)
model/action_tokenizer.py:390
↓ 1 callersMethodspherical_to_cartesian
(self, theta, phi, r)
model/action_tokenizer.py:99
↓ 1 callersFunctionsplit_to_even_chunks
Split a list of indices into `chunks` chunks of roughly equal lengths.
train/monkey_patch.py:78
↓ 1 callersMethodtie_weights
(self)
model/modeling_spatialvla.py:243
↓ 1 callersFunctionto_padding
(tensor: tf.Tensor)
data/utils/data_utils.py:38
↓ 1 callersFunctiontree_map
(fn: Callable, tree: Dict)
data/utils/data_utils.py:21
↓ 1 callersFunctiontree_merge
(*trees: Dict)
data/utils/data_utils.py:27
↓ 1 callersFunctionvelocity_act_to_wrist_frame
Translates velocity actions (translation + rotation) from base frame of the robot to wrist frame. Args: velocity: 6d velocity action
data/utils/droid_utils.py:36
Method__call__
Discretize continuous actions to tokens. action: np.ndarray, (n, 7), continuous actions in Cartesian or Spherical coordinates. return:
model/action_tokenizer.py:37
Method__call__
Discretize continuous actions to tokens. action: np.ndarray, (n, 3), continuous actions in Cartesian or Spherical coordinates. return:
model/action_tokenizer.py:105
Method__call__
Discretize continuous actions to tokens. action: np.ndarray, (n, 3), continuous actions in Cartesian or Spherical coordinates. return:
model/action_tokenizer.py:177
Method__call__
Discretize continuous actions to tokens. action: np.ndarray, (n,), continuous actions in Cartesian or Spherical coordinates. return: n
model/action_tokenizer.py:227
Method__call__
Discretize continuous actions to tokens. action: np.ndarray, (n, 7), continuous actions in Cartesian coordinates. return: np.ndarray,
model/action_tokenizer.py:305
Method__call__
( self, images: ImageInput = None, text: Union[TextInput, PreTokenizedInput, List[Text
model/processing_spatialvla.py:103
Method__del__
(self)
train/monkey_patch.py:358
Method__init__
( self, tokenizer: PreTrainedTokenizerBase, num_bins: int = 256, min_action: i
model/action_tokenizer.py:15
Method__init__
( self, tokenizer: PreTrainedTokenizerBase, num_bins: Dict, bin_policy: Option
model/action_tokenizer.py:60
Method__init__
( self, tokenizer: PreTrainedTokenizerBase, num_bins: Dict, bin_policy: Option
model/action_tokenizer.py:144
Method__init__
( self, tokenizer: PreTrainedTokenizerBase, num_bins: int = 2, array_begin_idx
model/action_tokenizer.py:209
Method__init__
set bin_policy if exist, otherwise, caculate bin_policy from gs_params or use uniform bin grids. gs_params: Optional[Dict], bin_policy
model/action_tokenizer.py:262
Method__init__
(self, dim: int, eps: float = 1e-6)
model/modeling_gemma2.py:61
Method__init__
(self, dim, max_position_embeddings=2048, base=10000, device=None)
model/modeling_gemma2.py:96
Method__init__
(self, config: Gemma2Config, layer_idx: Optional[int] = None)
model/modeling_gemma2.py:328
Method__init__
(self, config: Gemma2Config, layer_idx: Optional[int] = None)
model/modeling_gemma2.py:417
Method__init__
(self, config: Gemma2Config, layer_idx: Optional[int] = None)
model/modeling_gemma2.py:427
Method__init__
(self, config: Gemma2Config, layer_idx: int)
model/modeling_gemma2.py:437
Method__init__
(self, config: Gemma2Config)
model/modeling_gemma2.py:655
Method__init__
(self, config)
model/modeling_gemma2.py:891
Method__init__
(self, config)
model/modeling_gemma2.py:1110
Method__init__
(self, config)
model/modeling_gemma2.py:1206
Method__init__
( self, vision_config=None, text_config=None, ignore_index=-100, image
model/configuration_spatialvla.py:26
Method__init__
(self, config: SpatialVLAConfig)
model/modeling_spatialvla.py:122
Method__init__
(self, config: SpatialVLAConfig, vision_model=None, vision_zoe_model=None, projector_model=None, language_mode
model/modeling_spatialvla.py:163
Method__init__
( self, image_processor=None, tokenizer=None, chat_template=None, stat
model/processing_spatialvla.py:41
next →1–100 of 244, ranked by callers