Write the sequence of args, with no separators, to the console and output files (if you've configured an output file).

Write the sequence of args, with no separators, to the console and output files (if you've configured an output file).

Predict method

Predict method

Calculate discounted forward sum of a sequence at each point

Calculate discounted forward sum of a sequence at each point

Fit model to current data batch + previous data batch Args: x: features y: target

Run policy and collect data for a minimum of min_steps and min_episodes Args: env: ai gym environment policy: policy object with

Build and initialize TensorFlow graph

Construct TensorFlow graph, including loss function, init op and train op

Build and initialize TensorFlow graph

Construct TensorFlow graph, including loss function, init op and train op

Launch TensorFlow session and initialize variables

Launch TensorFlow session and initialize variables

Add to Graph: 1. KL divergence between old and new distributions 2. Entropy of present policy given states and action

Add to Graph: 1. KL divergence between old and new distributions 2. Entropy of present policy given states and action

Calculate log probabilities of a batch of observations & actions

Calculate log probabilities of a batch of observations & actions

Input placeholders

Input placeholders

Neural net for policy approximation function

Sample from distribution, given observation

Sample from distribution, given observation

Adds discounted sum of rewards to all time steps of all trajectories Args: trajectories: as returned by run_policy() gamma: disc

Add generalized advantage estimator. https://arxiv.org/pdf/1506.02438.pdf Args: trajectories: as returned by run_policy(), must incl

Adds estimated value to all time steps of all trajectories Args: trajectories: as returned by run_policy() val_func: object with

Args: trajectories: trajectories after processing by add_disc_sum_rew(), add_value(), and add_gae() Returns: 4-tuple of

Close TensorFlow session

Close TensorFlow session

Fit model to current data batch + previous data batch Args: x: features y: target

returns 2-tuple: (scale, offset)

returns 2-tuple: (scale, offset)

Initialize gym environment, return dimension of observation and action spaces. Args: env_name: str environment name (e.g. "Human

Log batch statistics

Log a value of some diagnostic Call this once for each diagnostic quantity, each iteration

Log a value of some diagnostic Call this once for each diagnostic quantity, each iteration

Main training loop Args: env_name: OpenAI Gym environment name, e.g. 'Hopper-v1' num_iterations: maximum number of iterations to

Run single episode with option to animate Args: env: ai gym environment policy: policy object with sample() method scale

Draw sample from policy distribution

Draw sample from policy distribution

Update running mean and variance (this is an exact method) Args: x: NumPy array, shape = (N, obs_dim) see: https://stats

Update policy based on observations, actions and advantages Args: observes: observations, shape = (N, obs_dim) actio

Update running mean and variance (this is an exact method) Args: x: NumPy array, shape = (N, obs_dim) see: https://stats

Write key-value pairs

Write key-value pairs

Write a sequence of other data (e.g. a logging message)

Write a sequence of other data (e.g. a logging message)