Returns the summed penalty by applying `regularizer` to the `weights_list`. Adding a regularization penalty over the layer weights and embedding weights can help prevent overfitting the training data. Regularization over layer biases is less common/useful, but assuming proper data preprocessi
(regularizer, weights_list=None)
| 167 | |
| 168 | |
| 169 | def apply_regularization(regularizer, weights_list=None): |
| 170 | """Returns the summed penalty by applying `regularizer` to the `weights_list`. |
| 171 | |
| 172 | Adding a regularization penalty over the layer weights and embedding weights |
| 173 | can help prevent overfitting the training data. Regularization over layer |
| 174 | biases is less common/useful, but assuming proper data preprocessing/mean |
| 175 | subtraction, it usually shouldn't hurt much either. |
| 176 | |
| 177 | Args: |
| 178 | regularizer: A function that takes a single `Tensor` argument and returns |
| 179 | a scalar `Tensor` output. |
| 180 | weights_list: List of weights `Tensors` or `Variables` to apply |
| 181 | `regularizer` over. Defaults to the `GraphKeys.WEIGHTS` collection if |
| 182 | `None`. |
| 183 | |
| 184 | Returns: |
| 185 | A scalar representing the overall regularization penalty. |
| 186 | |
| 187 | Raises: |
| 188 | ValueError: If `regularizer` does not return a scalar output, or if we find |
| 189 | no weights. |
| 190 | """ |
| 191 | if not weights_list: |
| 192 | weights_list = ops.get_collection(ops.GraphKeys.WEIGHTS) |
| 193 | if not weights_list: |
| 194 | raise ValueError('No weights to regularize.') |
| 195 | with ops.name_scope( |
| 196 | 'get_regularization_penalty', values=weights_list) as scope: |
| 197 | penalties = [regularizer(w) for w in weights_list] |
| 198 | penalties = [ |
| 199 | p if p is not None else constant_op.constant(0.0) for p in penalties |
| 200 | ] |
| 201 | for p in penalties: |
| 202 | if p.get_shape().ndims != 0: |
| 203 | raise ValueError('regularizer must return a scalar Tensor instead of a ' |
| 204 | 'Tensor with rank %d.' % p.get_shape().ndims) |
| 205 | |
| 206 | summed_penalty = math_ops.add_n(penalties, name=scope) |
| 207 | ops.add_to_collection(ops.GraphKeys.REGULARIZATION_LOSSES, summed_penalty) |
| 208 | return summed_penalty |