Denoising Auto-Encoder class (dA) A denoising autoencoders tries to reconstruct the input from a corrupted version of it by projecting it first in a latent space and reprojecting it afterwards back in the input space. Please refer to Vincent et al.,2008 for more details. If x is the
| 52 | |
| 53 | |
| 54 | class dA(object): |
| 55 | """Denoising Auto-Encoder class (dA) |
| 56 | |
| 57 | A denoising autoencoders tries to reconstruct the input from a corrupted |
| 58 | version of it by projecting it first in a latent space and reprojecting |
| 59 | it afterwards back in the input space. Please refer to Vincent et al.,2008 |
| 60 | for more details. If x is the input then equation (1) computes a partially |
| 61 | destroyed version of x by means of a stochastic mapping q_D. Equation (2) |
| 62 | computes the projection of the input into the latent space. Equation (3) |
| 63 | computes the reconstruction of the input, while equation (4) computes the |
| 64 | reconstruction error. |
| 65 | |
| 66 | .. math:: |
| 67 | |
| 68 | \tilde{x} ~ q_D(\tilde{x}|x) (1) |
| 69 | |
| 70 | y = s(W \tilde{x} + b) (2) |
| 71 | |
| 72 | x = s(W' y + b') (3) |
| 73 | |
| 74 | L(x,z) = -sum_{k=1}^d [x_k \log z_k + (1-x_k) \log( 1-z_k)] (4) |
| 75 | |
| 76 | """ |
| 77 | |
| 78 | def __init__( |
| 79 | self, |
| 80 | numpy_rng, |
| 81 | theano_rng=None, |
| 82 | input=None, |
| 83 | n_visible=784, |
| 84 | n_hidden=500, |
| 85 | W=None, |
| 86 | bhid=None, |
| 87 | bvis=None |
| 88 | ): |
| 89 | """ |
| 90 | Initialize the dA class by specifying the number of visible units (the |
| 91 | dimension d of the input ), the number of hidden units ( the dimension |
| 92 | d' of the latent or hidden space ) and the corruption level. The |
| 93 | constructor also receives symbolic variables for the input, weights and |
| 94 | bias. Such a symbolic variables are useful when, for example the input |
| 95 | is the result of some computations, or when weights are shared between |
| 96 | the dA and an MLP layer. When dealing with SdAs this always happens, |
| 97 | the dA on layer 2 gets as input the output of the dA on layer 1, |
| 98 | and the weights of the dA are used in the second stage of training |
| 99 | to construct an MLP. |
| 100 | |
| 101 | :type numpy_rng: numpy.random.RandomState |
| 102 | :param numpy_rng: number random generator used to generate weights |
| 103 | |
| 104 | :type theano_rng: theano.tensor.shared_randomstreams.RandomStreams |
| 105 | :param theano_rng: Theano random generator; if None is given one is |
| 106 | generated based on a seed drawn from `rng` |
| 107 | |
| 108 | :type input: theano.tensor.TensorType |
| 109 | :param input: a symbolic description of the input or None for |
| 110 | standalone dA |
| 111 |