Perform the CD-`k` "forward pass" of visible inputs into hidden units and back. Notes ----- This implementation follows [1]_'s recommendations for the RBM forward pass: - Use real-valued probabilities for both the data and the visible
(self, V, K=None, retain_derived=True)
| 466 | self.backward() |
| 467 | |
| 468 | def forward(self, V, K=None, retain_derived=True): |
| 469 | """ |
| 470 | Perform the CD-`k` "forward pass" of visible inputs into hidden units |
| 471 | and back. |
| 472 | |
| 473 | Notes |
| 474 | ----- |
| 475 | This implementation follows [1]_'s recommendations for the RBM forward |
| 476 | pass: |
| 477 | |
| 478 | - Use real-valued probabilities for both the data and the visible |
| 479 | unit reconstructions. |
| 480 | - Only the final update of the hidden units should use the actual |
| 481 | probabilities -- all others should be sampled binary states. |
| 482 | - When collecting the pairwise statistics for learning weights or |
| 483 | the individual statistics for learning biases, use the |
| 484 | probabilities, not the binary states. |
| 485 | |
| 486 | References |
| 487 | ---------- |
| 488 | .. [1] Hinton, G. (2010). "A practical guide to training restricted |
| 489 | Boltzmann machines". *UTML TR 2010-003* |
| 490 | |
| 491 | Parameters |
| 492 | ---------- |
| 493 | V : :py:class:`ndarray <numpy.ndarray>` of shape `(n_ex, n_in)` |
| 494 | Visible input, representing the `n_in`-dimensional features for a |
| 495 | minibatch of `n_ex` examples. Each feature in V should ideally be |
| 496 | binary-valued, although it is possible to also train on real-valued |
| 497 | features ranging between (0, 1) (e.g., grayscale images). |
| 498 | K : int |
| 499 | The number of steps of contrastive divergence steps to run before |
| 500 | computing the gradient update. If None, use ``self.K``. Default is |
| 501 | None. |
| 502 | retain_derived : bool |
| 503 | Whether to retain the variables calculated during the forward pass |
| 504 | for use later during backprop. If False, this suggests the layer |
| 505 | will not be expected to backprop through wrt. this input. Default |
| 506 | is True. |
| 507 | """ |
| 508 | if not self.is_initialized: |
| 509 | self.n_in = V.shape[1] |
| 510 | self._init_params() |
| 511 | |
| 512 | # override self.K if necessary |
| 513 | K = self.K if K is None else K |
| 514 | |
| 515 | W = self.parameters["W"] |
| 516 | b_in = self.parameters["b_in"] |
| 517 | b_out = self.parameters["b_out"] |
| 518 | |
| 519 | # compute hidden unit probabilities |
| 520 | Z_H = V @ W + b_out |
| 521 | p_H = self.act_fn_H.fn(Z_H) |
| 522 | |
| 523 | # sample hidden states (stochastic binary values) |
| 524 | H = np.random.rand(*p_H.shape) <= p_H |
| 525 | H = H.astype(float) |
no test coverage detected