Every trial, we randomly select two of the four cues to provide to the network. Every timestep within that trial we either randomly display only zeros, or we alternate between the two cues in the pair. At the end of a trial, we provide the response cue, for which th
(self, action)
| 37 | self.reset() # Simply reset according to grid definition. |
| 38 | |
| 39 | def step(self, action): |
| 40 | """ |
| 41 | Every trial, we randomly select two of the four cues to provide to the |
| 42 | network. Every timestep within that trial we either randomly display |
| 43 | only zeros, or we alternate between the two cues in the pair. |
| 44 | |
| 45 | At the end of a trial, we provide the response cue, for which the network |
| 46 | must respond 1 if the target was one of the provided cues or 0 if it was |
| 47 | not. The next timestep, we evaluate the response, giving a reward of 1 |
| 48 | for correct and -1 for incorrect. |
| 49 | |
| 50 | :param action: network's decision if the target cue was displayed. |
| 51 | :return obs: observation of vector with binary cue and the following fields: |
| 52 | - time since start of episode |
| 53 | - one-hot-encoded value for a response of 0 in previous timestep |
| 54 | - one-hot-encoded value for a response of 1 in previous timestep |
| 55 | - reward of previous timestep |
| 56 | :return reward: 1 for correct response; -1 for incorrect response. |
| 57 | :return done: indicates termination of simulation |
| 58 | :return info: dictionary including values for debugging purposes. |
| 59 | """ |
| 60 | self.tstep += 1 # increment episode timestep |
| 61 | self.trialTime -= 1 # decrement current trial timestep |
| 62 | |
| 63 | # Populate base fields of observation. |
| 64 | self.obs = self.zeroArray # default to empty array |
| 65 | self.obs[-4] = self.tstep # time since start of episode. |
| 66 | self.obs[-3] = int(self.response == 0) # response = 0 for previous timestep |
| 67 | self.obs[-2] = int(self.response == 1) # response = 1 for previous timestep |
| 68 | self.obs[-1] = self.reward[0] # reward of previous timestep |
| 69 | |
| 70 | self.response = action # Remember previous response |
| 71 | self.reward[0] = 0 # default current reward to 0 |
| 72 | |
| 73 | # If starting a new trial |
| 74 | if self.trialTime <= 0: |
| 75 | # Set new trial length, based on mean number of trials per episode = 15 |
| 76 | self.trialTime = random.randint(10, 20) // self.ep_duration |
| 77 | |
| 78 | # Randomly select the pair of cues to be shown to the network. |
| 79 | self.pairmask = np.array(range(NUM_CUES))[ |
| 80 | np.argsort(np.random.uniform(0, 1, 4)) < 2 |
| 81 | ] |
| 82 | |
| 83 | # Determine if target is one of these current cues displayed. |
| 84 | self.targ_disp = int(np.any(self.pairmask == self.target)) |
| 85 | |
| 86 | self.cue_pair_ind = 0 # Reset cue pair indicator |
| 87 | |
| 88 | # Deterministic special cases for last two trial timesteps. |
| 89 | if self.trialTime <= 2: |
| 90 | # If it's the second to last trial timestep, cue response. |
| 91 | # Response cue is another binary cue vector but with a value of 1. |
| 92 | if self.trialTime == 2: |
| 93 | self.obs[0] = 1 |
| 94 | |
| 95 | # If it's the last trial timestep, provide empty input, check |
| 96 | # check the answer to the response cue, and compute the reward. |