Deep Belief Network A deep belief network is obtained by stacking several RBMs on top of each other. The hidden layer of the RBM at layer `i` becomes the input of the RBM at layer `i+1`. The first layer RBM gets as input the input of the network, and the hidden layer of the last RBM
| 18 | |
| 19 | # start-snippet-1 |
| 20 | class DBN(object): |
| 21 | """Deep Belief Network |
| 22 | |
| 23 | A deep belief network is obtained by stacking several RBMs on top of each |
| 24 | other. The hidden layer of the RBM at layer `i` becomes the input of the |
| 25 | RBM at layer `i+1`. The first layer RBM gets as input the input of the |
| 26 | network, and the hidden layer of the last RBM represents the output. When |
| 27 | used for classification, the DBN is treated as a MLP, by adding a logistic |
| 28 | regression layer on top. |
| 29 | """ |
| 30 | |
| 31 | def __init__(self, numpy_rng, theano_rng=None, n_ins=784, |
| 32 | hidden_layers_sizes=[500, 500], n_outs=10): |
| 33 | """This class is made to support a variable number of layers. |
| 34 | |
| 35 | :type numpy_rng: numpy.random.RandomState |
| 36 | :param numpy_rng: numpy random number generator used to draw initial |
| 37 | weights |
| 38 | |
| 39 | :type theano_rng: theano.tensor.shared_randomstreams.RandomStreams |
| 40 | :param theano_rng: Theano random generator; if None is given one is |
| 41 | generated based on a seed drawn from `rng` |
| 42 | |
| 43 | :type n_ins: int |
| 44 | :param n_ins: dimension of the input to the DBN |
| 45 | |
| 46 | :type hidden_layers_sizes: list of ints |
| 47 | :param hidden_layers_sizes: intermediate layers size, must contain |
| 48 | at least one value |
| 49 | |
| 50 | :type n_outs: int |
| 51 | :param n_outs: dimension of the output of the network |
| 52 | """ |
| 53 | |
| 54 | self.sigmoid_layers = [] |
| 55 | self.rbm_layers = [] |
| 56 | self.params = [] |
| 57 | self.n_layers = len(hidden_layers_sizes) |
| 58 | |
| 59 | assert self.n_layers > 0 |
| 60 | |
| 61 | if not theano_rng: |
| 62 | theano_rng = MRG_RandomStreams(numpy_rng.randint(2 ** 30)) |
| 63 | |
| 64 | # allocate symbolic variables for the data |
| 65 | |
| 66 | # the data is presented as rasterized images |
| 67 | self.x = T.matrix('x') |
| 68 | |
| 69 | # the labels are presented as 1D vector of [int] labels |
| 70 | self.y = T.ivector('y') |
| 71 | # end-snippet-1 |
| 72 | # The DBN is an MLP, for which all weights of intermediate |
| 73 | # layers are shared with a different RBM. We will first |
| 74 | # construct the DBN as a deep multilayer perceptron, and when |
| 75 | # constructing each sigmoidal layer we also construct an RBM |
| 76 | # that shares weights with that layer. During pretraining we |
| 77 | # will train these RBMs (which will lead to chainging the |