MCPcopy Create free account
hub / github.com/google-research/language / create_model

Function create_model

language/conpono/reconstruct/model_builder.py:35–164  ·  view source on GitHub ↗

Creates a classification model. Args: model: the BERT model from modeling.py labels: ground truth paragraph order decoder_inputs: the input to the decoder if used batch_size: the batch size model_type: one of decode, pooled, attn sep_positions: (optional) for "pooled" inde

(model,
                 labels,
                 decoder_inputs,
                 batch_size,
                 model_type="decode",
                 sep_positions=None)

Source from the content-addressed store, hash-verified

33
34
35def create_model(model,
36 labels,
37 decoder_inputs,
38 batch_size,
39 model_type="decode",
40 sep_positions=None):
41 """Creates a classification model.
42
43 Args:
44 model: the BERT model from modeling.py
45 labels: ground truth paragraph order
46 decoder_inputs: the input to the decoder if used
47 batch_size: the batch size
48 model_type: one of decode, pooled, attn
49 sep_positions: (optional) for "pooled" indecies of SEP tokens
50
51 Returns:
52 tuple of (loss, per_example_loss, logits, probabilities) for model
53 """
54
55 output_layer = model.get_pooled_output()
56
57 hidden_size = output_layer.shape[-1].value
58 tpu_batch_size = tf.shape(output_layer)[0]
59
60 num_labels = 5 # GOOGLE-INTERNAL TODO(daniter) this shouldn't be hardcoded
61
62 with tf.variable_scope("paragraph_reconstruct"):
63 if model_type == "decode":
64 lstm_cell = tf.nn.rnn_cell.LSTMCell(
65 num_units=hidden_size, use_peepholes=True, state_is_tuple=True)
66
67 def sample_fn(x):
68 return tf.to_float(tf.reshape(tf.argmax(x, axis=-1), (-1, 1)))
69
70 helper = FixedSizeInferenceHelper(
71 sample_fn=sample_fn,
72 sample_shape=[1],
73 sample_dtype=tf.float32,
74 start_inputs=decoder_inputs[:, 0],
75 end_fn=None)
76
77 # Decoder
78 project_layer = tf.layers.Dense(
79 num_labels, use_bias=False, name="output_projection")
80
81 my_decoder = contrib_seq2seq.BasicDecoder(
82 lstm_cell,
83 helper,
84 tf.nn.rnn_cell.LSTMStateTuple(output_layer, output_layer),
85 output_layer=project_layer)
86
87 # Dynamic decoding
88 outputs, _, _ = contrib_seq2seq.dynamic_decode(
89 my_decoder,
90 swap_memory=True,
91 scope="paragraph_reconstruct",
92 maximum_iterations=5)

Callers

nothing calls this directly

Calls 4

constantMethod · 0.80
get_pooled_outputMethod · 0.45
get_sequence_outputMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…