This function encapsulates the inference and the learning. If the agent is in train mode (mode = -1): It starts by callin

Returns a cell from the cells list. @param x cell x coordinate @param y cell y coordinate @returns cell

Handle the command line and return an object containing all the parameters. Arguments: args - list of command line arguments (not inc

Activate this controller. All controllers inheriting this class should call this method in their own __init()__ using super(self.__c

Update a leaf and the tree priorities. When the replay memory is updated with a new transition, it is also updated in the tree. The

Build a network consistent with each type of inputs

Gets the number of different actions that can be taken on this environment. It can be either an integer in the case of a finite discrete numbe

Get all observations currently in the replay memory, ordered by time where they were observed.

Returns the index corresponding to a timestep that is valid

Instantiate a Keras model for fitting a float from x. The model takes the following inputs: x : internal state

Gets the shape of the input space for this environment. This returns a list whose length is the number of observations in the environ

Set the target Q-network weights equal to the main Q-network weights

Get all actions currently in the replay memory, ordered by time where they were taken.

Clip the possible actions if it is outside the action space defined by self._nActions self._nActions is given as [[low_action1,high

Returns the average sum of rewards per episode and the number of episode

Instantiate a a Keras model for the Q-network from x. The model takes the following inputs: x : internal state Parameters

Compile self.q_vals

Build balanced subtrees. The leaf nodes have their "priority" initialized to 0 and "position" from 0 to n-1, with n being the size

Set the target Q-network weights equal to the main Q-network weights

This function runs an episode of learning. An episode ends up when the environment method "inTerminalState" returns True (or when th

This function runs a number of epochs in non train mode (id > -1). Parameters ----------- n_epochs : int

Simulate one time step in the environment and returns the reward for the time step Parameters ----------- action : l

Main method of the Policy class. It can be called by agent.py, given a state, and should return a valid action w.r.t. the environment given to

Store the punctual observations, action, reward, is_terminal and priority in the dataset. Parameters ----------- obs : ndarra

Returns the best Action for the given state. This is an additional encapsulation for q-network.

Instantiate a Keras model that provides the difference between two encoded pseudo-states The model takes the two following inputs:

Instantiate a Keras model for the encoder of the CRAR learning algorithm. The model takes the following as input s : list o

Optional hook called at the end of all epochs

Instantiate a a Keras model for the Q-network from s. The model takes the following inputs: s : list of objects Each ob

Instantiate a Keras model for fitting a float from s. The model takes the four following inputs: s : list of objects

Generate the indices of a random batch of size n. The samples within the random batch are selected following the priorities (probabil

Obtain the high-dimensional observation from indices of the agent position and the indices of the reward positions.

Called when the agent has finished processing all its epochs, just before returning from its run() method.

Called whenever the agent ends an epoch, just after the last episode of this epoch was ended and after any onEpisodeEnd() signal was processe

Called when the agent is going to start working (before anything else). This corresponds to the moment where the agent's run() method

Returns a random action

Setting the discount factor Parameters ----------- df : float The discount factor that has to bet set

Set the epsilon used for :math:`\epsilon`-greedy exploration

Setting the learning rate Parameters ----------- lr : float The learning rate that has to be set

Instantiate a Keras model for the transition between two encoded pseudo-states. The model takes as inputs: x : internal state

Update adjacent cell. @param adj adjacent cell to current cell @param cell current cell being processed

Avoid terminal states in the x samples preceding the chosen index. Argument: index - chosen index based on prio

Compile all the optimizers for the different losses

This function encapsulates the whole process of the learning. It starts by calling the controllers method "onStart", Then it

This method is called at each time step and performs one action in the environment. Returns ------- V : float

Applies the agent action [action] on the environment. Parameters ----------- action : int The action selected by

Returns the average training loss on the epoch

Returns the average V value on the episode (on time steps where a non-random action has been taken)

Get the best action for a pseudo-state

For plan_depth=0, instantiate a Keras model that provides the difference between T(E(s1),a) and E(s2). Note that it gives 0 if the transition

Getting the discount factor

Dump the network Parameters ----------- fname : string Name of the file where the network will be dumped

Get the epsilon for :math:`\epsilon`-greedy exploration

Find a leaf based on the priority. Arguments: priority - the target priority generated randomly Return: nod

Find a leaf based on the index. Arguments: index - integer between 0 and n-1, n being the size of the repl

Instantiate a Keras model that provides the vector of the transition at E(s1). It is calculated as the different between E(s1) and E(T(s1)).

Provides all parameters used by the learning algorithm Returns ------- Values of the parameters: list of numpy arrays

Returns adjacent cells to a cell. Clockwise starting from the one on the right. @param cell get adjacent cells for this cell @

Compute the heuristic value H for a cell. Distance between this cell and the ending cell multiply by 10. @returns heuristic value H