Updates graph with quantization operations. Currently we quantize the following tensors: * Conv/MatMul: Quantize the weights if it matches. * Activation: Quantize the output if it matches. * Bypass/Post-activation Bypass: Quantize both input and output if it matches. Args: graph:
(graph,
is_training,
per_channel_wt=False,
per_channel_act=False,
weight_bits=8,
activation_bits=8,
symmetric=False,
ema_decay=0.999,
quant_delay=None,
vars_collection=ops.GraphKeys.GLOBAL_VARIABLES,
scope=None,
use_qdq=False)
| 45 | |
| 46 | |
| 47 | def Quantize(graph, |
| 48 | is_training, |
| 49 | per_channel_wt=False, |
| 50 | per_channel_act=False, |
| 51 | weight_bits=8, |
| 52 | activation_bits=8, |
| 53 | symmetric=False, |
| 54 | ema_decay=0.999, |
| 55 | quant_delay=None, |
| 56 | vars_collection=ops.GraphKeys.GLOBAL_VARIABLES, |
| 57 | scope=None, |
| 58 | use_qdq=False): |
| 59 | """Updates graph with quantization operations. |
| 60 | |
| 61 | Currently we quantize the following tensors: |
| 62 | * Conv/MatMul: Quantize the weights if it matches. |
| 63 | * Activation: Quantize the output if it matches. |
| 64 | * Bypass/Post-activation Bypass: Quantize both input and output |
| 65 | if it matches. |
| 66 | |
| 67 | Args: |
| 68 | graph: Graph to modify. |
| 69 | is_training: Whether quantizing training graph or eval graph. |
| 70 | weight_bits: Number of bits to use for quantizing weights. |
| 71 | activation_bits: Number of bits to use for quantizing activations. |
| 72 | symmetric: (Optional) If true, use symmetric quantization limits instead of |
| 73 | training the minimum and maximum of each quantization range separately. |
| 74 | ema_decay: (Optional) Float, EMA decay parameter. EMA is used to update |
| 75 | quantization intervals for quantizing activations (see here about EMA: |
| 76 | https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average). |
| 77 | quant_delay: (Optional, default None) Int, count of global steps for which |
| 78 | to delay quantization. This helps weights stabilize at the start of |
| 79 | training. |
| 80 | vars_collection: (Optional) Collection where to store the variables for |
| 81 | quantization interval ends. |
| 82 | scope: The scope to be transformed. If it's not None, only the ops which |
| 83 | are in this scope will be transformed. |
| 84 | use_qdq: Use tf.quantize_and_dequantize_v3 (qdq) op instead of fake_quant_with_min_max_vars |
| 85 | for quantization. The qdq op is used for scaling with no zero point. |
| 86 | Raises: |
| 87 | ValueError: When quantization fails. |
| 88 | """ |
| 89 | if scope and not scope.endswith('/'): |
| 90 | scope += '/' |
| 91 | |
| 92 | input_to_ops_map = input_to_ops.InputToOps(graph) |
| 93 | quantized_ops = set() |
| 94 | for layer_match in _FindLayersToQuantize(graph): |
| 95 | # Quantize the weights. |
| 96 | context = _GetContextFromOp(layer_match.layer_op) |
| 97 | |
| 98 | # If `scope` is given, only quantize it if the consumer of weights |
| 99 | # (the layer op) is in the right scope. |
| 100 | if layer_match.weight_tensor is not None: |
| 101 | _InsertQuantOp( |
| 102 | context, |
| 103 | 'weights_quant', |
| 104 | layer_match.weight_tensor.op, |
nothing calls this directly
no test coverage detected