MCPcopy Create free account
hub / github.com/google-research/language / compute_average_embs

Method compute_average_embs

language/emql/model.py:457–482  ·  view source on GitHub ↗

Compute the text encoding of questions. We take a bag of word approach. Question encoding is an average pooling of word embeddings. Args: question: a list of token ids Returns: a tensor for question encoding

(self, question)

Source from the content-addressed store, hash-verified

455 return topk_fact_loss, tensors
456
457 def compute_average_embs(self, question):
458 """Compute the text encoding of questions.
459
460 We take a bag of word approach. Question encoding is an average pooling
461 of word embeddings.
462
463 Args:
464 question: a list of token ids
465
466 Returns:
467 a tensor for question encoding
468 """
469 question_mask = tf.cast(tf.not_equal(question, -1), tf.int32)
470 # batch_size, max_question_len
471 question_mask_3d = tf.cast(
472 tf.expand_dims(question_mask, axis=2), tf.float32)
473 question_embs = tf.nn.embedding_lookup(
474 self.word_embeddings_mat, question * question_mask)
475 # batch_size, max_question_len, hidden_size
476 question_embs = tf.reduce_sum(question_embs * question_mask_3d, axis=1)
477 # batch_size, hidden_size
478 question_len = tf.reduce_sum(question_mask, axis=1, keepdims=True)
479 # batch_size, 1
480 question_embs /= tf.cast(question_len, tf.float32)
481 # batch_size, hidden_size
482 return question_embs
483
484 def model_webqsp(self, features, params,
485 top_k):

Callers 1

model_metaqaMethod · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected