MCPcopy Create free account
hub / github.com/deepspeedai/DeepSpeed / quantize_tenary

Method quantize_tenary

deepspeed/runtime/quantize.py:102–113  ·  view source on GitHub ↗
(self, inputs)

Source from the content-addressed store, hash-verified

100 return output
101
102 def quantize_tenary(self, inputs):
103 input_flat = inputs.reshape(self.q_groups, -1)
104 n = input_flat.shape[1]
105 m = input_flat.norm(p=1, dim=1).div(n)
106 thres = (0.7 * m).view(-1, 1) #.expand_as(input_flat)
107 pos = (input_flat > thres).type(inputs.type())
108 neg = (input_flat < -thres).type(inputs.type())
109 mask = (input_flat.abs() > thres).type(inputs.type())
110 alpha = ((mask * input_flat).abs().sum(dim=1) / mask.sum(dim=1)).view(-1, 1)
111 output = alpha * pos - alpha * neg
112 output = output.reshape(inputs.shape).contiguous()
113 return output
114
115 def quantize_binary(self, inputs):
116 input_flat = inputs.reshape(self.q_groups, -1)

Callers 1

compute_quantizationMethod · 0.95

Calls 3

typeMethod · 0.80
contiguousMethod · 0.80
reshapeMethod · 0.45

Tested by

no test coverage detected