Initialize a history block, stack the previous stacked_observations frames. Parameters ---------- init_observations: list

Prepare the observations to satisfy the input fomat of torch [B, S, W, H, C] -> [B, S x C, W, H] batch, stack num, width, height, channel

Do MCTS for the roots (a batch of root nodes in parallel). Parallel in model inference Parameters ---------- roots: Any

MLP layers Parameters ---------- input_size: int dim of inputs layer_sizes: list dim of hidden layers output_size:

To obtain an observation of correct format: o[t, t + stack frames + extra len] Parameters ---------- i: int time s

put the last game history into the pool if the current game is finished Parameters ---------- last_game_histories: list

evaluation test Parameters ---------- model: any models for evaluation counter: int current training step counter

To reduce memory usage, we choose to store the jpeg strings of image instead of the numpy array in the buffer. This function encodes the observati

Consistency loss function: similarity loss Parameters

Reference from MuZerp: Appendix F => Network Architecture & Appendix A : Proposition A.2 in https://arxiv.org/pdf/1805.11593.pdf (Page-11)

Make Atari games Parameters ---------- env_id: str name of environment skip: int frame skip max_episode_steps: int

returns a new instance of the game

Reference from MuZerp: Appendix F => Network Architecture & Appendix A : Proposition A.2 in https://arxiv.org/pdf/1805.11593.pdf (Page-11)

select action from the root visit counts. Parameters ---------- temperature: float the temperature for the distribution determ

To reduce memory usage, we choose to store the jpeg strings of image instead of the numpy array in the buffer. This function decodes the observati

update models given a batch data Parameters ---------- model: Any EfficientZero models batch: Any a batch data inlcude

prepare policy targets from the non-reanalyzed context of policies

prepare the context of policies for non-reanalyzing part, just return the policy in self-play Parameters ---------- indices: l

prepare policy targets from the reanalyzed context of policies

prepare the context of policies for reanalyzing part Parameters ---------- indices: list transition index in repla

prepare reward and value targets from the context of rewards and values

prepare the context of rewards and values for reanalyzing part Parameters ---------- indices: list transition inde

training loop Parameters ---------- model: Any EfficientZero models target_model: Any EfficientZero models for reanaly

prepare the context of a batch reward_value_context: the context of reanalyzed value targets policy_re_context: the c

To make sure the correction of value targets, we need to add (o_t, r_t, etc) from the next history block , which is necessary for the bootstra