Stacked denoising auto-encoder class (SdA) A stacked denoising autoencoder model is obtained by stacking several dAs. The hidden layer of the dA at layer `i` becomes the input of the dA at layer `i+1`. The first layer dA gets as input the input of the SdA, and the hidden layer of th
| 49 | |
| 50 | # start-snippet-1 |
| 51 | class SdA(object): |
| 52 | """Stacked denoising auto-encoder class (SdA) |
| 53 | |
| 54 | A stacked denoising autoencoder model is obtained by stacking several |
| 55 | dAs. The hidden layer of the dA at layer `i` becomes the input of |
| 56 | the dA at layer `i+1`. The first layer dA gets as input the input of |
| 57 | the SdA, and the hidden layer of the last dA represents the output. |
| 58 | Note that after pretraining, the SdA is dealt with as a normal MLP, |
| 59 | the dAs are only used to initialize the weights. |
| 60 | """ |
| 61 | |
| 62 | def __init__( |
| 63 | self, |
| 64 | numpy_rng, |
| 65 | theano_rng=None, |
| 66 | n_ins=784, |
| 67 | hidden_layers_sizes=[500, 500], |
| 68 | n_outs=10, |
| 69 | corruption_levels=[0.1, 0.1] |
| 70 | ): |
| 71 | """ This class is made to support a variable number of layers. |
| 72 | |
| 73 | :type numpy_rng: numpy.random.RandomState |
| 74 | :param numpy_rng: numpy random number generator used to draw initial |
| 75 | weights |
| 76 | |
| 77 | :type theano_rng: theano.tensor.shared_randomstreams.RandomStreams |
| 78 | :param theano_rng: Theano random generator; if None is given one is |
| 79 | generated based on a seed drawn from `rng` |
| 80 | |
| 81 | :type n_ins: int |
| 82 | :param n_ins: dimension of the input to the sdA |
| 83 | |
| 84 | :type hidden_layers_sizes: list of ints |
| 85 | :param hidden_layers_sizes: intermediate layers size, must contain |
| 86 | at least one value |
| 87 | |
| 88 | :type n_outs: int |
| 89 | :param n_outs: dimension of the output of the network |
| 90 | |
| 91 | :type corruption_levels: list of float |
| 92 | :param corruption_levels: amount of corruption to use for each |
| 93 | layer |
| 94 | """ |
| 95 | |
| 96 | self.sigmoid_layers = [] |
| 97 | self.dA_layers = [] |
| 98 | self.params = [] |
| 99 | self.n_layers = len(hidden_layers_sizes) |
| 100 | |
| 101 | assert self.n_layers > 0 |
| 102 | |
| 103 | if not theano_rng: |
| 104 | theano_rng = RandomStreams(numpy_rng.randint(2 ** 30)) |
| 105 | # allocate symbolic variables for the data |
| 106 | self.x = T.matrix('x') # the data is presented as rasterized images |
| 107 | self.y = T.ivector('y') # the labels are presented as 1D vector of |
| 108 | # [int] labels |