Optimization parameters for Adagrad. Args: learning_rate: used for updating embedding table. initial_accumulator: initial accumulator for Adagrad. use_gradient_accumulation: setting this to `False` makes embedding gradients calculation less accurate but faster. Please
(self,
learning_rate,
initial_accumulator=0.1,
use_gradient_accumulation=True,
clip_weight_min=None,
clip_weight_max=None)
| 291 | """ |
| 292 | |
| 293 | def __init__(self, |
| 294 | learning_rate, |
| 295 | initial_accumulator=0.1, |
| 296 | use_gradient_accumulation=True, |
| 297 | clip_weight_min=None, |
| 298 | clip_weight_max=None): |
| 299 | """Optimization parameters for Adagrad. |
| 300 | |
| 301 | Args: |
| 302 | learning_rate: used for updating embedding table. |
| 303 | initial_accumulator: initial accumulator for Adagrad. |
| 304 | use_gradient_accumulation: setting this to `False` makes embedding |
| 305 | gradients calculation less accurate but faster. Please see |
| 306 | `optimization_parameters.proto` for details. |
| 307 | for details. |
| 308 | clip_weight_min: the minimum value to clip by; None means -infinity. |
| 309 | clip_weight_max: the maximum value to clip by; None means +infinity. |
| 310 | """ |
| 311 | super(AdagradParameters, |
| 312 | self).__init__(learning_rate, use_gradient_accumulation, |
| 313 | clip_weight_min, clip_weight_max) |
| 314 | if initial_accumulator <= 0: |
| 315 | raise ValueError('Adagrad initial_accumulator must be positive') |
| 316 | self.initial_accumulator = initial_accumulator |
| 317 | |
| 318 | |
| 319 | @tf_export(v1=['tpu.experimental.AdamParameters']) |