MCPcopy Create free account
hub / github.com/HA-Transformer/MAT / BeamableMM

Class BeamableMM

fairseq/modules/beamable_mm.py:10–47  ·  view source on GitHub ↗

This module provides an optimized MM for beam decoding with attention. It leverage the fact that the source-side of the input is replicated beam times and the target-side of the input is of width one. This layer speeds up inference by replacing the inputs {(bsz x 1 x nhu), (bsz x sz2 x

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 1

make_generation_fast_Method · 0.90

Calls

no outgoing calls

Tested by

no test coverage detected