Flatten inputs along dimension 1 and then pass through MLP.

Usage: ``` path_builder = PathBuilder() path.add_sample( observations=1, actions=2, next_observations=3,

Usage: ``` policy = ReparamTanhMultivariateGaussianPolicy(...) action, mean, log_std, _ = policy(obs) action, mean, log_std, _ =

My reparameterized normal implementation

version that: - uses reparameterization trick - has two Q functions - has auto-tuned alpha

Represent distribution of X where X ~ tanh(Z) Z ~ MultivariateNormal(mean, log_sig_diag)

This class pickles data into files and unpickles data from files. TODO: Handle logging text to terminal, GUI text, and/or log file at

Take a random discrete action with some probability.

Hindsight Experience Replay. Default using TD3 for RL policy trainer.

Simple 1D LayerNorm.

Custom for Ant Rand Goal The only difference is that it linearly embeds the goal into a higher dimension

Normalize action to in [-1, 1]. Optionally normalize observations and scale reward.

Policy with encoder Usage: ``` policy = ReparamTanhMultivariateGaussianEncoderPolicy(...)

Calulates the running mean and std of a data stream. https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm

Vectorized environment wrapper based on subprocess.

Twin Delayed Deep Deterministic policy gradients https://arxiv.org/abs/1802.09477

Depending on choice of reward function and size of replay buffer this will be: - AIRL - GAIL (without extra entropy term)

Depending on choice of reward function and size of replay buffer this will be: - AIRL - GAIL (without extra entropy term)

Usage: ``` policy = ReparamTanhMultivariateGaussianPolicy(...) action, mean, log_std, _ = policy(obs) action, mean, log_std, _ =

A cloudpickle wrapper used in SubprocVectorEnv.

Note about implementation: We will copy over the expert replay buffer into self.replay_buffer and when sampling batches, we will sample t

Usage: ``` policy = DiscretePolicy(...) action, log_prob = policy(obs, return_log_prob=True) ```

version that: - uses reparameterization trick - has two Q functions and a V function TODO: Recently in rlkit there is a versi

Dummy vectorized environment wrapper, implemented in for-loop. .. seealso:: Please refer to :class:`~tianshou.env.BaseVectorEnv` for other

Goal conditioned supervised learning https://github.com/dibyaghosh/gcsl https://arxiv.org/abs/1912.06088 Using goal-conditioned val

Hindsight Experience Replay. Default using TD3 for RL policy trainer.

Policy that always outputs zero.

Usage: ``` policy = ReparamTanhMultivariateGaussianPolicy(...) action, mean, log_std, _ = policy(obs) action, mean, log_std, _ =

version that: - uses reparameterization trick - has two Q functions - has auto-tuned alpha

Scale the obs if desired Unscale the acts if desired

Wrapper of multiprocessing Array.

THE MAX LENGTH OF AN EPISODE SHOULD BE STRICTLY SMALLER THAN THE max_replay_buffer_size OTHERWISE THERE IS A BUG IN TERMINATE_EPISODE #

SAC-AE with RAD and CURL for image-based tasks. version that: - uses reparameterization trick - has two Q functions -

version that: - uses reparameterization trick - has two Q functions and a V function TODO: Recently in rlkit there is a versi

Subprocess worker used in SubprocVectorEnv and ShmemVectorEnv.

Usage: vg = VariantGenerator() vg.add("param1", [1, 2, 3]) vg.add("param2", ['x', 'y']) vg.variants() => # all combinations of [

base algorithm for single task setting can be used for RL or Learning from Demonstrations

Base class for vectorized environments wrapper. Usage: :: env_num = 8 envs = DummyVectorEnv([lambda: gym.make(task) for _ in r

Deep Deterministic Policy Gradient

Dummy worker used in sequential vector environments.

An abstract worker for an environment.

A class used to save and replay data.

This strategy adds Gaussian noise to the action taken by the deterministic policy. Based on the rllab implementation.

COM inertia (cinert), COM velocity (cvel), actuator forces (qfrc_actuator), and external forces (cfrc_ext) are removed from the observation.