MCPcopy Create free account
hub / github.com/google-research/language / encoder

Function encoder

language/labs/exemplar_decoding/models/baselines.py:62–225  ·  view source on GitHub ↗

Model function. Atttention seq2seq model, augmented with an encoder over the targets of the nearest neighbors. Args: features: Dictionary of input Tensors. mode: train or eval. Keys from tf.estimator.ModeKeys. vocab: A list of strings of words in the vocabulary. hps: Hyperpar

(features, mode, vocab, hps)

Source from the content-addressed store, hash-verified

60
61
62def encoder(features, mode, vocab, hps):
63 """Model function.
64
65 Atttention seq2seq model, augmented with an encoder
66 over the targets of the nearest neighbors.
67
68 Args:
69 features: Dictionary of input Tensors.
70 mode: train or eval. Keys from tf.estimator.ModeKeys.
71 vocab: A list of strings of words in the vocabulary.
72 hps: Hyperparams.
73
74 Returns:
75 Encoder outputs.
76 """
77
78 # [batch_size, src_len]
79 src_inputs = features["src_inputs"]
80 src_len = features["src_len"]
81
82 with tf.variable_scope("embeddings"):
83 embeddings = tf.get_variable(
84 "embeddings",
85 [vocab.size(), hps.emb_dim],
86 dtype=tf.float32,
87 initializer=tf.uniform_unit_scaling_initializer())
88
89 # [batch_size, src_len, emb_dim]
90 src_encoder_input_emb = tf.nn.embedding_lookup(embeddings, src_inputs)
91 if mode == tf_estimator.ModeKeys.TRAIN and hps.emb_drop > 0.:
92 src_encoder_input_emb = tf.nn.dropout(
93 src_encoder_input_emb, keep_prob=1.0-hps.emb_drop)
94
95 src_att_context, neighbor_att_context = None, None
96 src_copy_context, neighbor_copy_context = None, None
97 with tf.variable_scope("src_encoder"):
98
99 # 2 * [batch_size, src_len, encoder_dim]
100 src_encoder_outputs, src_encoder_states = tf.nn.bidirectional_dynamic_rnn(
101 cell_fw=get_rnn_cell(
102 mode=mode, hps=hps,
103 input_dim=hps.emb_dim,
104 num_units=hps.encoder_dim,
105 num_layers=hps.num_encoder_layers,
106 dropout=hps.encoder_drop,
107 cell_type=hps.rnn_cell),
108 cell_bw=get_rnn_cell(
109 mode=mode, hps=hps,
110 input_dim=hps.emb_dim,
111 num_units=hps.encoder_dim,
112 num_layers=hps.num_encoder_layers,
113 dropout=hps.encoder_drop,
114 cell_type=hps.rnn_cell),
115 inputs=src_encoder_input_emb,
116 dtype=tf.float32,
117 sequence_length=src_len)
118
119 # [batch_size, src_len, 2*encoder_dim]

Callers

nothing calls this directly

Calls 3

get_rnn_cellFunction · 0.90
_build_contextFunction · 0.70
sizeMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…