MCPcopy Create free account
hub / github.com/tensorflow/lingvo / SetupRoutingTransformerEncoder

Function SetupRoutingTransformerEncoder

lingvo/core/routing_config_helper.py:51–148  ·  view source on GitHub ↗

A lightweight wrapper for Routing Transformer encoder stack. Args: model_dim: specifies dimension of transformer layers, token embeddings, and positional embeddings as well context vectors (attention values). hidden_dim: hidden dim of model. num_layers: number of transformer layers.

(model_dim,
                                   hidden_dim,
                                   num_layers,
                                   num_heads,
                                   left_context,
                                   right_context,
                                   block_size,
                                   num_routing_layers,
                                   num_routing_heads,
                                   num_clusters,
                                   attention_window,
                                   atten_dropout_prob=0.,
                                   relu_dropout_prob=0.,
                                   residual_dropout_prob=0.)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers

nothing calls this directly

Calls 3

SetMethod · 0.95
ParamsMethod · 0.45
SetMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…