Save a game history block Parameters ---------- game: Any a game history block end_tag: bool T

training process Parameters ---------- summary_writer: Any logging for tensorboard model_path: str model path for resu

Base Config for EfficietnZero Parameters ---------- training_steps: int training steps while collecting data

Queue storage Parameters ---------- threshold: int if the current size if larger than threshold, the data won't be

Shared storage for models and others Parameters ---------- model: any models for self-play (update every checkpoin

Data Worker for collecting data through self-play Parameters ---------- rank: int id of the worker replay_

Linear interpolation between initial_p and final_p over schedule_timesteps. After this many timesteps pass final_p is returned.

Sample initial states by taking random number of no-ops on reset. No-op is assumed to be action 0.

Make end-of-life == end-of-episode, but only reset on true game over. Done by DeepMind for the DQN and co. since it helps value estimation.

Return only every `skip`-th frame

Warp frames to 84x84 as done in the Nature paper and later work. If the environment uses dictionary observations, `dict_space_key` ca

Parameters ---------- action_space: int action space max_length: int max transition number of

CPU Batch Worker for reanalyzing targets, see Appendix. Prepare the context concerning CPU overhead Parameters ----------

GPU Batch Worker for reanalyzing targets, see Appendix. receive the context from CPU maker and deal with GPU overheads Parameters

Base Network schedule_timesteps. After this many timesteps pass final_p is returned. Parameters ---------- inv

Atari Wrapper Parameters ---------- env: Any another env wrapper discount: float discount of e

Representation network Parameters ---------- observation_shape: tuple or list shape of observations: [C, W, H]

Dynamics network Parameters ---------- num_blocks: int number of res blocks num_channels: int

Prediction network Parameters ---------- action_space_size: int action space num_blocks: int n

EfficientZero network Parameters ---------- observation_shape: tuple or list shape of observations: [C, W, H]

Prepare a batch context that contains: game_lst: a list of game histories game_pos_lst: transition index in ga

Do no-op action for a number of steps in [1, noop_max].

Reset only when lives are exhausted. This way all states are still reachable even though lives are episodic, and the learner need not

Repeat action, sum reward, and max over last observations.