MCPcopy Create free account
hub / github.com/Turing-Project/WriteGPT / dropout

Function dropout

LanguageNetwork/GPT2/train/utils.py:130–144  ·  view source on GitHub ↗

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keeping* a dimension as in `tf.nn.dropout`). Returns: A version of `input_tensor` with dropout applied.

(input_tensor, dropout_prob)

Source from the content-addressed store, hash-verified

128
129
130def dropout(input_tensor, dropout_prob):
131 """Perform dropout.
132
133 Args:
134 input_tensor: float Tensor.
135 dropout_prob: Python float. The probability of dropping out a value (NOT of
136 *keeping* a dimension as in `tf.nn.dropout`).
137
138 Returns:
139 A version of `input_tensor` with dropout applied.
140 """
141 if dropout_prob is None or dropout_prob == 0.0:
142 return input_tensor
143 output = tf.nn.dropout(input_tensor, rate=dropout_prob)
144 return output
145
146
147def get_attention_mask(nd, ns, *, dtype):

Callers 2

attention_layerFunction · 0.90
residual_mlp_layerFunction · 0.90

Calls

no outgoing calls

Tested by

no test coverage detected