Adds a decay to weights in the model. This is a form of L2 regularization. Args: weight_decay: strength of the regularization
(model, weight_decay)
| 2182 | |
| 2183 | |
| 2184 | def add_weight_decay(model, weight_decay): |
| 2185 | """Adds a decay to weights in the model. |
| 2186 | |
| 2187 | This is a form of L2 regularization. |
| 2188 | |
| 2189 | Args: |
| 2190 | weight_decay: strength of the regularization |
| 2191 | """ |
| 2192 | _build( |
| 2193 | model, |
| 2194 | WeightDecayBuilder(weight_decay=weight_decay), |
| 2195 | weights_only=True, |
| 2196 | use_param_info_optim=False, |
| 2197 | ) |
| 2198 | |
| 2199 | |
| 2200 | def build_sgd( |
searching dependent graphs…