MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / GetTolerance

Function GetTolerance

tensorflow/lite/kernels/div_test.cc:74–79  ·  view source on GitHub ↗

For quantized Div, the error shouldn't exceed (2*step + step^2).

Source from the content-addressed store, hash-verified

72
73// For quantized Div, the error shouldn't exceed (2*step + step^2).
74inline float GetTolerance(int min, int max) {
75 const float kQuantizedStep = (max - min) / 255.0f;
76 const float kQuantizedTolerance =
77 2.0f * kQuantizedStep + kQuantizedStep * kQuantizedStep;
78 return kQuantizedTolerance;
79}
80
81TEST(FloatDivOpTest, NoActivation) {
82 FloatDivOpModel m({TensorType_FLOAT32, {1, 2, 2, 1}},

Callers 4

QuantizedNoActivationFunction · 0.70
QuantizedWithBroadcastFunction · 0.70

Calls

no outgoing calls

Tested by

no test coverage detected