MCPcopy
hub / github.com/Turing-Project/WriteGPT / dropout

Function dropout

LanguageNetwork/GPT2/scripts/utils.py:128–142  ·  view source on GitHub ↗

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keeping* a dimension as in `tf.nn.dropout`). Returns: A version of `input_tensor` with dropout applied.

(input_tensor, dropout_prob)

Source from the content-addressed store, hash-verified

126
127
128def dropout(input_tensor, dropout_prob):
129 """Perform dropout.
130
131 Args:
132 input_tensor: float Tensor.
133 dropout_prob: Python float. The probability of dropping out a value (NOT of
134 *keeping* a dimension as in `tf.nn.dropout`).
135
136 Returns:
137 A version of `input_tensor` with dropout applied.
138 """
139 if dropout_prob is None or dropout_prob == 0.0:
140 return input_tensor
141 output = tf.nn.dropout(input_tensor, rate=dropout_prob)
142 return output
143
144
145def get_attention_mask(nd, ns, *, dtype):

Callers 2

attention_layerFunction · 0.90
residual_mlp_layerFunction · 0.90

Calls

no outgoing calls

Tested by

no test coverage detected