Return the navigable viewpoints as a string.

Make action and update the agent's state.

Evaluate each agent trajectory based on how close it got to the goal location the path contains [view_id, angle, vofv]

Get the agent's candidate list from pre-stored navigable dict.

Return the history of actions taken.

Interface between Panoramic view and Egocentric view Take in the next viewpoint ID and move the agent to that viewpoint retur

Return a list of dicts containing instr_id:'xx', path:[(viewpointId, heading_rad, elevation_rad)]

Construct the scratchpad that lets the agent continue its thought process.

Create a tool to back trace during navigation. The tool is invoked with the history of navigation trajectory. Using the LLM to find a

Create a tool to make single action prediction in MP3D. The tool is invoked with the simulation environment and records the action ta

load graph from self.scan, Store the graph {scan_id: graph} in self.graphs Store the shortest path {scan_id: {view_id_x: {vie

Store the minibach in 'self.batch'

Instantiate API planner and controller for a given trajectory. We use a top-level "orchestrator" agent to invoke the planner and controller,

Euclidean distance between two graph poses

Construct the pipeline object from model_id and task.

Get list of states augmented with precomputed image features. rgb field will be empty. Agent's current view [0-35] (set only when vie

Return the history of visited viewpoints for back tracing.

Initialize the trajectory with the given observation.

Load parameters for the rendezvous distributed procedure

Load connectivity graph for each scan

Take an action using the full state dependent action interface (with batched input)

:param x: start :param y: end :return: the path from x to y [v1, v2, ..., v_n, y]

Load a new minibatch / episodes.

Take action (same interface as makeActions)

Initialize the LLM Navigation Agent. Args: env: The Matterport3D environment. config: The configuration.

1. Load pretrained image feature 2. Init the Simulator. :param feat_db: The name of file stored the feature. :param b

Back trace the action plan.

Get the identifying parameters.

Make single step action in MatterSim.

Run all_gather on arbitrary picklable data (not necessarily tensors) Args: data: any picklable object Returns: list[data]

Create the full inputs for the LLMChain from intermediate steps.