Returns a function which differentiates f with respect to variables. The wrapped function returns the value and the gradient of f when called with the same arguments. The gradient is with respect to all trainable TFE variables accessed by `f`. This function is useful when the exact set of
(f)
| 150 | |
| 151 | |
| 152 | def implicit_val_and_grad(f): |
| 153 | """Returns a function which differentiates f with respect to variables. |
| 154 | |
| 155 | The wrapped function returns the value and the gradient of f when called with |
| 156 | the same arguments. The gradient is with respect to all trainable TFE |
| 157 | variables accessed by `f`. |
| 158 | |
| 159 | This function is useful when the exact set of variables to differentiate with |
| 160 | is not known ahead of time. |
| 161 | |
| 162 | Example: |
| 163 | |
| 164 | ```python |
| 165 | dense_layer = tf.compat.v1.layers.Dense(1) |
| 166 | def loss(x, y): |
| 167 | return tf.reduce_sum(tf.square(dense_layer(x) - y)) |
| 168 | |
| 169 | # Obtain the gradient function. |
| 170 | val_grad_fn = tfe.implicit_value_and_gradients(loss) |
| 171 | |
| 172 | # Invoke the gradient function with concrete values of x and y. |
| 173 | x = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) |
| 174 | y = tf.constant([[10.0], [20.0]]) |
| 175 | value, grads_and_vars = val_grad_fn(x, y) |
| 176 | print('Value of loss: %s' % value) |
| 177 | |
| 178 | # Apply the gradients to Variables. |
| 179 | optimizer = tf.compat.v1.train.GradientDescentOptimizer(0.1) |
| 180 | optimizer.apply_gradients(grads_and_vars) |
| 181 | ``` |
| 182 | |
| 183 | Args: |
| 184 | f: function to be differentiated. If `f` returns a scalar, this scalar will |
| 185 | be differentiated. If `f` returns a tensor or list of tensors, by default |
| 186 | a scalar will be computed by adding all their values to produce a single |
| 187 | scalar. |
| 188 | |
| 189 | Returns: |
| 190 | A function which, when called, returns a tuple pair. |
| 191 | Its first element is the value to which the function evaluates. |
| 192 | Its second element is list of (gradient, variable) pairs. |
| 193 | |
| 194 | Raises: |
| 195 | ValueError: if `f` returns None. |
| 196 | """ |
| 197 | # TODO(cais): Remove calls to tf.constant() once the gradients functions |
| 198 | # accept lists and np.ndarrays. |
| 199 | |
| 200 | def grad_fn(*args, **kwds): |
| 201 | """Computes the gradient of the wrapped function.""" |
| 202 | this_tape = tape.push_new_tape() |
| 203 | try: |
| 204 | end_node = f(*args, **kwds) |
| 205 | if end_node is None: |
| 206 | raise ValueError("Cannot differentiate a function that returns None; " |
| 207 | "did you forget to return a value from {}?".format( |
| 208 | f.__name__)) |
| 209 | finally: |