MCPcopy Create free account
hub / github.com/tensorflow/lingvo / TransformerEncoderLayer

Method TransformerEncoderLayer

lingvo/core/batch_major_attention.py:9434–9473  ·  view source on GitHub ↗

(inputs, paddings) -> (encoded, paddings). Args: name: the string name of the encoder layer params. stride: To use every k-th token, set the stride to k. When stride == 0, only returns the first token of the input. When stride == 1, returns every token in the input.

(self, name, stride=1, first_n=None,
                              ff_hidden_dim=None, num_heads=None,
                              use_moe=False, layer_idx=None)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Calls 4

MoEMethod · 0.95
FeedforwardMethod · 0.95
_StridedAttentionMethod · 0.95
_SeqMethod · 0.45