MCPcopy Create free account
hub / github.com/HA-Transformer/MAT / TransformerDecoderLayer

Class TransformerDecoderLayer

fairseq/modules/transformer_layer.py:117–271  ·  view source on GitHub ↗

Decoder layer block. In the original paper each operation (multi-head attention, encoder attention or FFN) is postprocessed with: `dropout -> add residual -> layernorm`. In the tensor2tensor code they suggest that learning is more robust when preprocessing each layer with layernorm

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 3

__init__Method · 0.90
__init__Method · 0.90
__init__Method · 0.90

Calls

no outgoing calls

Tested by

no test coverage detected