MCPcopy Create free account
hub / github.com/XiaoMi/mace / quantize_int16

Function quantize_int16

tools/python/quantize/quantize_util.py:242–256  ·  view source on GitHub ↗
(data)

Source from the content-addressed store, hash-verified

240
241# only support int16 symmetric quantization.
242def quantize_int16(data):
243 np_data = np.array(data).astype(float)
244 max_val = max(abs(np_data.min()), abs(np_data.max()))
245 scale = max_val / 2**15
246 zero = 0
247 output = np.clip((np.round(zero + data / scale).astype(np.int32)),
248 -2**15, 2**15 - 1)
249
250 quantized_data = QuantizedData()
251 quantized_data.data = output
252 quantized_data.scale = scale
253 quantized_data.zero = zero
254 quantized_data.minval = -max_val
255 quantized_data.maxval = max_val
256 return quantized_data
257
258
259def quantize_bias_for_hexagon(data):

Callers

nothing calls this directly

Calls 4

QuantizedDataClass · 0.85
roundMethod · 0.80
maxFunction · 0.50
absFunction · 0.50

Tested by

no test coverage detected