MCPcopy Create free account
hub / github.com/TeleHuman/TextOp / quantization

Method quantization

TextOpRobotMDAR/robotmdar/train/manager.py:327–336  ·  view source on GitHub ↗
(tensor, bits=8)

Source from the content-addressed store, hash-verified

325
326 @staticmethod
327 def quantization(tensor, bits=8):
328 if bits == 8:
329 scale = 127.0 / tensor.abs().max().clamp(min=1e-8)
330 quantized = (tensor * scale).round() / scale
331 elif bits == 16:
332 quantized = tensor.half().float()
333 else:
334 quantized = tensor
335
336 return quantized
337
338 @staticmethod
339 def quat_geodesic_loss(q_pred, q_target):

Callers 1

calc_geometry_lossMethod · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected