Identify which call to tf.gradients created this gradient op or tensor. TensorArray gradient calls use an accumulator TensorArray object. If multiple gradients are calculated and run in the same session, the multiple gradient nodes may accidentally flow throuth the same accumulator TensorArr
(op_or_tensor)
| 40 | |
| 41 | |
| 42 | def _GetGradSource(op_or_tensor): |
| 43 | """Identify which call to tf.gradients created this gradient op or tensor. |
| 44 | |
| 45 | TensorArray gradient calls use an accumulator TensorArray object. If |
| 46 | multiple gradients are calculated and run in the same session, the multiple |
| 47 | gradient nodes may accidentally flow throuth the same accumulator TensorArray. |
| 48 | This double counting breaks the TensorArray gradient flow. |
| 49 | |
| 50 | The solution is to identify which gradient call this particular |
| 51 | TensorArray*Grad is being called in, by looking at the input gradient |
| 52 | tensor's name, and create or lookup an accumulator gradient TensorArray |
| 53 | associated with this specific call. This solves any confusion and ensures |
| 54 | different gradients from the same forward graph get their own accumulators. |
| 55 | |
| 56 | This function creates the unique label associated with the tf.gradients call |
| 57 | that is used to create the gradient TensorArray. |
| 58 | |
| 59 | Args: |
| 60 | op_or_tensor: `Tensor` or `Operation` which is an input to a |
| 61 | TensorArray*Grad call. |
| 62 | |
| 63 | Returns: |
| 64 | A python string, the unique label associated with this particular |
| 65 | gradients calculation. |
| 66 | |
| 67 | Raises: |
| 68 | ValueError: If not called within a gradients calculation. |
| 69 | """ |
| 70 | name_tokens = op_or_tensor.name.split("/") |
| 71 | grad_pos = [i for i, x in enumerate(name_tokens) if x.startswith("gradients")] |
| 72 | if not grad_pos: |
| 73 | raise ValueError( |
| 74 | "Expected op/tensor name to start with gradients (excluding scope)" |
| 75 | ", got: {}. This means that a tf.gradients op with this op in its " |
| 76 | "dependency path has a custom name that does not start with " |
| 77 | "'gradients'. Please make sure all calls to tf.gradients that have " |
| 78 | "non-empty 'name' arguments use names that start with " |
| 79 | "'gradients'.".format(op_or_tensor.name)) |
| 80 | return "/".join(name_tokens[:grad_pos[-1] + 1]) |
| 81 | |
| 82 | |
| 83 | @ops.RegisterGradient("TensorArrayRead") |
no test coverage detected