MCPcopy Create free account
hub / github.com/XiaoMi/mace / quantize_int8

Function quantize_int8

tools/python/quantize/quantize_util.py:184–204  ·  view source on GitHub ↗
(data)

Source from the content-addressed store, hash-verified

182
183
184def quantize_int8(data):
185 np_data = np.array(data).astype(float)
186 in_min = np_data.min()
187 in_max = np_data.max()
188
189 in_min = min(0.0, in_min)
190 in_max = max(0.0, in_max)
191 max_abs = max(abs(in_min), abs(in_max))
192 zero = 0
193 scale = max_abs / 127
194
195 output = np.clip((np.round(zero + np_data / scale).astype(np.int32)),
196 -127, 127)
197
198 quantized_data = QuantizedData()
199 quantized_data.data = output
200 quantized_data.scale = scale
201 quantized_data.zero = zero
202 quantized_data.minval = -127 * scale
203 quantized_data.maxval = 127 * scale
204 return quantized_data
205
206
207def quantize_with_min_and_max(data, device, non_zero, in_min, in_max):

Callers

nothing calls this directly

Calls 5

QuantizedDataClass · 0.85
roundMethod · 0.80
minFunction · 0.50
maxFunction · 0.50
absFunction · 0.50

Tested by

no test coverage detected