Compute a recurrent neural net. Roughly, Recurrent() computes the following: state = state0 for t in inputs' sequence length: state = cell_fn(theta, state, inputs[t, :]) accumulate_state[t, :] = state return accumulate_state, state theta, state, inputs are all structure
(theta,
state0,
inputs,
cell_fn,
cell_grad=None,
extras=None,
max_input_length=None,
use_tpu=False,
aligned_end=False)
| 653 | |
| 654 | |
| 655 | def Recurrent(theta, |
| 656 | state0, |
| 657 | inputs, |
| 658 | cell_fn, |
| 659 | cell_grad=None, |
| 660 | extras=None, |
| 661 | max_input_length=None, |
| 662 | use_tpu=False, |
| 663 | aligned_end=False): |
| 664 | """Compute a recurrent neural net. |
| 665 | |
| 666 | Roughly, Recurrent() computes the following: |
| 667 | state = state0 |
| 668 | for t in inputs' sequence length: |
| 669 | state = cell_fn(theta, state, inputs[t, :]) |
| 670 | accumulate_state[t, :] = state |
| 671 | return accumulate_state, state |
| 672 | |
| 673 | theta, state, inputs are all structures of tensors. |
| 674 | |
| 675 | inputs[t, :] means taking a slice out from every tensor in the inputs. |
| 676 | |
| 677 | accumulate_state[t, :] = state means that we stash every tensor in |
| 678 | 'state' into a slice of the corresponding tensor in |
| 679 | accumulate_state. |
| 680 | |
| 681 | cell_fn is a python callable computing (building up a TensorFlow |
| 682 | graph) the recurrent neural network's one forward step. Two calls of |
| 683 | cell_fn must describe two identical computations. |
| 684 | |
| 685 | By construction, Recurrent()'s backward computation does not access |
| 686 | any intermediate values computed by cell_fn during forward |
| 687 | computation. We may extend Recurrent() to support that by taking a |
| 688 | customized backward function of cell_fn. |
| 689 | |
| 690 | Args: |
| 691 | theta: weights. A structure of tensors. |
| 692 | state0: initial state. A structure of tensors. |
| 693 | inputs: inputs. A structure of tensors. |
| 694 | cell_fn: A python function, which computes: |
| 695 | state1, extras = cell_fn(theta, state0, inputs[t, :]) |
| 696 | cell_grad: A python function which computes: |
| 697 | dtheta, dstate0, dinputs[t, :] = cell_grad( |
| 698 | theta, state0, inputs[t, :], extras, dstate1) |
| 699 | extras: A structure of tensors. The 2nd return value of every |
| 700 | invocation of cell_fn is a structure of tensors with matching keys |
| 701 | and shapes of this `extras`. |
| 702 | max_input_length: maximum length of effective input. This is used to |
| 703 | truncate the computation if the inputs have been allocated to a |
| 704 | larger size. A scalar tensor. |
| 705 | use_tpu: whether or not we are on TPU. |
| 706 | aligned_end: A boolean indicating whether the sequence is aligned at |
| 707 | the end. |
| 708 | |
| 709 | Returns: |
| 710 | accumulate_state and the final state. |
| 711 | """ |
| 712 | if cell_grad is None and _IsSingleTimeStep(inputs, max_input_length): |
nothing calls this directly
no test coverage detected