MCPcopy Create free account
hub / github.com/google-research/language / encoder

Function encoder

language/labs/exemplar_decoding/models/hypernet.py:69–302  ·  view source on GitHub ↗

Model function. Atttention seq2seq model, augmented with an encoder over the targets of the nearest neighbors. Args: features: Dictionary of input Tensors. mode: train or eval. Keys from tf.estimator.ModeKeys. vocab: A list of strings of words in the vocabulary. hps: Hyperpar

(features, mode, vocab, hps)

Source from the content-addressed store, hash-verified

67
68
69def encoder(features, mode, vocab, hps):
70 """Model function.
71
72 Atttention seq2seq model, augmented with an encoder
73 over the targets of the nearest neighbors.
74
75 Args:
76 features: Dictionary of input Tensors.
77 mode: train or eval. Keys from tf.estimator.ModeKeys.
78 vocab: A list of strings of words in the vocabulary.
79 hps: Hyperparams.
80
81 Returns:
82 Encoder outputs.
83 """
84
85 # [batch_size, src_len]
86 src_inputs = features["src_inputs"]
87 src_len = features["src_len"]
88
89 with tf.variable_scope("embeddings"):
90 scale = (3.0 / hps.emb_dim) ** 0.5
91 embeddings = tf.get_variable(
92 "embeddings",
93 [vocab.size(), hps.emb_dim],
94 dtype=tf.float32,
95 initializer=tf.random_uniform_initializer(minval=-scale, maxval=scale))
96
97 # [batch_size, src_len, emb_dim]
98 src_input_emb = tf.nn.embedding_lookup(embeddings, src_inputs)
99
100 if mode == tf_estimator.ModeKeys.TRAIN and hps.emb_drop > 0.:
101 src_input_emb = tf.nn.dropout(
102 src_input_emb, keep_prob=1.0-hps.emb_drop)
103 src_att_context, neighbor_att_context = None, None
104 src_copy_context, neighbor_copy_context = None, None
105 with tf.variable_scope("src_encoder"):
106
107 # 2 * [batch_size, src_len, encoder_dim]
108 src_encoder_outputs, src_encoder_states = tf.nn.bidirectional_dynamic_rnn(
109 cell_fw=get_rnn_cell(
110 mode=mode, hps=hps,
111 input_dim=hps.emb_dim,
112 num_units=hps.encoder_dim,
113 num_layers=hps.num_encoder_layers,
114 dropout=hps.encoder_drop,
115 cell_type="lstm"),
116 cell_bw=get_rnn_cell(
117 mode=mode, hps=hps,
118 input_dim=hps.emb_dim,
119 num_units=hps.encoder_dim,
120 num_layers=hps.num_encoder_layers,
121 dropout=hps.encoder_drop,
122 cell_type="lstm"),
123 inputs=src_input_emb,
124 dtype=tf.float32,
125 sequence_length=src_len)
126

Callers 1

bodyMethod · 0.50

Calls 3

get_rnn_cellFunction · 0.90
_build_contextFunction · 0.70
sizeMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…