Loads a config from a file.

Loads model from an experiment folder.

Outputs the current reward mean

Launches an agent process with agent_id Args: agent_id (int): agent's id iterations (int): if not none,

Launches multiple agent processes with agent_id in agent_ids Useful when you want agents to share a GPU Args:

Launches an eval processes with id eval_id Args: eval_id (int): eval agent's id mode: eval_deterministic

Launches multiple eval processes with agent_id in agent_ids Useful when you want agents to share a GPU Args:

Launches the learner process. Learner consumes experience from replay and publishes experience to parameter serve

Launches a loggerplex server. It helps distributed logging.

Lauches the parameter server process. Serves parameters to agents

Launches the replay process. Replay collects experience from agents and serve them to learner

Launches the learner and agent facing load balancing proxys for replays

Launches a single replay server Args: replay_id: The id of the replay server

Launches a tensorboard process

Launches a tensorplex process. It receives data from multiple sources and send them to tensorboard.

returning the running obseravtion mean for Tensorplex logging Returns: numpy array of current running observa

returning the running square mean for Tensorplex Logging Returns: running square mean

returning the running standard deviation for Tensorplex Logging Returns: numpy array of running standard devi

save to `<name>.<global_steps>.ckpt` file Args: score: float scalar to be compared, for `keep_best` global_s

Save config into a yaml file in root experiment directory

Given uint8 input from the environment, scale to float32 and divide by 255 to scale inputs between 0.0 and 1.0

Sends the current experiences to distributed replay buffer Args: data: list of sub-trajectories collected

We can improve this function if we *really* need more memory efficiency

queue of (score, filepath) tuples

Sets up the related states for running components Args: args: A list of commandline arguments provided.

Tells the thread to start sampling only when this condition is met. For example, only when the replay memory has > 10K experiences.

stops recording and wait for the video_process to finish writing to file and join First puts a None (End of Video) frame into the fra

Value of the schedule at time t

updates Z_filter running obs mean and variance Args: obs: dictionary of batched observations

Get the current estimate

delegate any other methods to the underlying parser

Value remains constant over time. Parameters ---------- value: float Constant value of the schedule

Piecewise schedule. endpoints: [(int, int)] list of pairs `(time, value)` meanining that schedule should output `valu

Linear interpolation between initial_p and final_p over schedule_timesteps. After this many timesteps pass final_p is returned.

Same signature as Checkpoint Args: period: Only update when the interval counter % period == 0 min_interval:

Args: decay: Decay factor of smoothed moving average Default is 0.9995, which is approximately moving average

Figures out what docker images need to be built Populates self.images_provided which computes {name: identifi

Args: num_agents: number of agents to run num_evals: number of evals to run executable: path to .py execu

Setup an surreal experiment Args: agent_class: The Agent subclass to run for agents learner_class: The Agent

Args: port: the port connected to the pub socket module_dict: ModuleDict object that exposes model parameters

Args: publisher_host, publisher_port: where learner publish parameters serving_host, serving_port: where to serve par

Args: host: parameter server host port: parameter server port timeout: how long should the the client wai

Args: flush_iteration: how many send() calls before we flush the buffer

is_uint8: scale down uint8 image input from Atari by 255.0 You can use atari_wrapper.ScaledFloatFrame, but dividing on GPU is more

Constructor for RewardFilter class Args: eps: tolerance value for computing reward filter (whitening)

Constructor for ZFilter class Args: obs_spec: nested dictionary of observation space spec. see doc eps: toler

Constructor for PPO actor network Args: D_obs: observation space dimension, scalar D_act: ac

Constructor for PPO critic network Args: D_obs: observation space dimension, scalar hidden_s

first: if True, triggers at the first time