hub / github.com/tensorflow/lingvo / ExtendStep

Method ExtendStep

lingvo/core/layers.py:3528–3556 · view source on GitHub ↗

Generates a JTensor of sinusoids with different frequencies for a step. Args: theta: A `.NestedMap` object containing weights' values of this layer and its children layers. inputs: The input sequence on which to apply the Rotary position embedding. Since rotary posit

(self, theta, inputs, time_step)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 15

testExtendStepAsyncTimeStepSelfAttentionMethod · 0.45

testMultipleExtendStepAsyncTimeStepSelfAttentionMethod · 0.45

testExtendStepMethod · 0.45

testExtendStepAsyncTimeStepSelfAttentionMethod · 0.45

testExtendStepSelfAttentionMethod · 0.45

testExtendStepMethod · 0.45

testTransformerAttentionLayerExtendStepMethod · 0.45

testTransformerAttentionLayerExtendStepMixHeadsMethod · 0.45

testTransformerAttentionLayerNoLayernormMethod · 0.45

testTransformerLayerFPropDtypesMethod · 0.45

testTransformerLayerDecodeWithCrossAttentionMethod · 0.45

Calls 1

FPropMethod · 0.95

Tested by 15

testExtendStepAsyncTimeStepSelfAttentionMethod · 0.36

testMultipleExtendStepAsyncTimeStepSelfAttentionMethod · 0.36

testExtendStepMethod · 0.36

testExtendStepAsyncTimeStepSelfAttentionMethod · 0.36

testExtendStepSelfAttentionMethod · 0.36

testExtendStepMethod · 0.36

testTransformerAttentionLayerExtendStepMethod · 0.36

testTransformerAttentionLayerExtendStepMixHeadsMethod · 0.36

testTransformerAttentionLayerNoLayernormMethod · 0.36

testTransformerLayerFPropDtypesMethod · 0.36

testTransformerLayerDecodeWithCrossAttentionMethod · 0.36