Create the optimizer and logging operations for a single agent. Parameters ---------- scope : str the outer scope

Create the optimizer and logging operations for a single agent. Parameters ---------- scope : str an outer scope

Create the cooperative gradients meta-policy optimizer.

Store a transition in the replay buffer. Parameters ---------- obs0 : array_like the last observation act

Update the storage of observations for individual vehicles. This method performs the following operation: 1. It adds the most recent

Update the positions and speeds of all vehicles. Parameters ---------- pos : array_like positions of all vehicles

Add a new transition to the buffer. Parameters ---------- obs_t : array_like the list of environment observations

Check if n_samples samples can be sampled from the buffer. Returns ------- bool True if enough sample exist, Fals

Choice the x array with the smallest number of elements.

Compute the environment reward. This is defined by the child classes.

See parent class.

See parent class.

Create the image corresponding to the current state.

Compute fraction of variance that ypred explains about y. Returns 1 - Var[y-ypred] / Var[y] interpretation: ev=0 => might as well

Calculate the gradient and flattens it. Parameters ---------- loss : float the loss value var_list : list of tf.Tensor

Return the global variables. Parameters ---------- name : str the scope Returns ------- list of tf.Variable

See parent class.

Return a black and white image.

Return the orientation of the ant.

Return the relative observation of a vehicle. The observation consists of (by index): 1. the ego speed 2. the headway 3. the speed o

Get state, which concatenates joint positions and velocities.

Convert a batch to a td_map.

Import an environment from the flow/examples folder. This method imports the flow_params dict from the exp_configs folders in this directory

Check whether a policy is a multi-agent policy.

Return the intersection of Line(pt1,pt2) and Line(ptA,ptB). Taken from https://www.cs.hmc.edu/ACM/lectures/intersections.html Parameters

Return the logp of an action from the old or current policy.

Create an actor tensor. Parameters ---------- obs : tf.compat.v1.placeholder the input observation placeholder

Create an actor tensor. Parameters ---------- obs : tf.compat.v1.placeholder the input observation placeholder

Create an actor tensor. Parameters ---------- obs : tf.compat.v1.placeholder the input observation placeholder of

Create a critic tensor. Parameters ---------- obs : tf.compat.v1.placeholder the input observation placeholder

Compute the position of the agent's center of mass.

Compute the position of the agent's center of mass.

Get TensorFlow Tensor's number of elements. Parameters ---------- tensor : tf.Tensor the input tensor Returns -------

Process a minibatch of samples. This method re-formats the data to numpy arrays that can be passed to the tensorflow placeholders, and comput

Multiply two quaternion.

Multiply two quaternion.

See parent class. In addition, a few variables that are specific to this class are emptied before they are used by the new rollout.

See parent class.

Advance the simulation by one step.

Test that the observed vehicles in the environment are as expected. Parameters ---------- env_class : flow.envs.Env class the env

Return the next position by the agent. Parameters ---------- state : array_like the state by the agent ac

Update the position of the camera.

Update the RL lists of controllable, entering, and exiting vehicles. Used by the open environments. Parameters ---------- env : flow

Update the camera position and orientation.

See parent class.

Initialize the UR5 environment. Parameters ---------- use_contexts : bool, optional specifies whether to add cont

Replace a portion of vehicles with automated vehicles.

Clear all AV-related attributes.

Perform the gradient update procedure for the CHER algorithm. This procedure is similar to update_from_batch, expect it runs the self

Convert the indices to appropriate samples.

Convert the indices to appropriate samples.

Return the starting position of all agents. Returns ------- list of (float, float) coordinates that the agen

Return the starting position of all agents. Returns ------- list of (float, float) coordinates that the agents ar

Check that there is an agent in the current maze structure.

Check that there is an agent in the current maze structure.

Compute the accelerations of individual vehicles. The acceleration values are dictated by the Intelligent Driver Model (IDM), which c

See get_action.

See get_action.

See get_td_map.

See get_td_map.

Calculate hindsight goal and action transitions. These are then stored in the replay buffer along with the original (non-hindsight) s

See initialize.

See initialize.

Check whether the agent is in a collision location. Parameters ---------- pos : (float, float) (x,y) positio

Check whether the agent is in a collision location. Parameters ---------- pos : (float, float) (x,y) position of

Calculate the log probability of the next goal by the meta-policies. Parameters ---------- meta_actions : array_like

Log training statistics. Parameters ---------- file_path : str the list of cumulative rewards from every episode

Return True if the intrinsic reward returns negative values. Intrinsic reward functions with negative rewards incentivize early termi

Return the negative-logp of the current policy.

Return the negative-logp of the previous policy.

Return True if the agent is in a wall.

Reset highway and/or I-210 style environments. New inflow and downstream speed limit conditions are chosen at the start of a new simu

Reset ring-style environments. If the "ring_length" term is specified, this method will update the network's length to a new value wi

Sample different goals. The goals are sampled as follows: * The first num_samples-2 goals are acquired from a random Gaussian

Return meta-actions that approximately maximize low-level log-probs. Parameters ---------- meta_obs0 : array_like

Create the actor loss, gradient, and optimizer.

Create minimization operations for policy and entropy. Creates a `tf.optimizer.minimize` operations for updating policy and entropy w

Create the actor loss and optimization process. Parameters ---------- all_obs_ph : tf.compat.v1.placeholder the p

Create the actor and alpha loss and optimization process. Parameters ---------- qf1_pi : tf.Variable the output f

Create basic independent learners / shared policy components. In this case, the policy consists of separate (or shared) policies for

Create the critic loss, gradient, and optimizer.

Create minimization operation for critic Q-function. Create a `tf.optimizer.minimize` operation for updating critic Q-function with g

Create the critic loss and optimization process. Parameters ---------- critic : tf.Variable the output from the c

Create the critic loss and optimization process. Parameters ---------- qf1_pi : tf.Variable the output from the f

Create algorithmic-variant of MADDPG components. See: https://arxiv.org/pdf/1706.02275.pdf

Perform independent form of MADDPG setup.

Perform independent form of MADDPG setup.

Perform shared form of MADDPG setup.

Perform shared form of MADDPG setup.

Create the actor and critic optimizers.

Create the actor and critic optimizers.

Create the running means and std of the model inputs and outputs. This method also adds the same running means and stds as scalars to

Create the running means and std of the model inputs and outputs. This method also adds the same running means and stds as scalars to

Create the running means and std of the model inputs and outputs. This method also adds the same running means and stds as scalars to

Create the running means and std of the model inputs and outputs. This method also adds the same running means and stds as scalars to

Prepare tensorboard logging for attributes of the agent. Parameters ---------- rew_ph : tf.compat.v1.placeholder

Prepare tensorboard logging for attributes of the agent. Parameters ---------- rew_ph : tf.compat.v1.placeholder