MCPcopy
hub / github.com/pquerna/ffjson / roundShortest

Function roundShortest

fflib/v1/ftoa.go:228–324  ·  view source on GitHub ↗

Round d (= mant * 2^exp) to the shortest number of digits that will let the original floating point value be precisely reconstructed. Size is original floating point size (64 or 32).

(d *decimal, mant uint64, exp int, flt *floatInfo)

Source from the content-addressed store, hash-verified

226// that will let the original floating point value be precisely
227// reconstructed. Size is original floating point size (64 or 32).
228func roundShortest(d *decimal, mant uint64, exp int, flt *floatInfo) {
229 // If mantissa is zero, the number is zero; stop now.
230 if mant == 0 {
231 d.nd = 0
232 return
233 }
234
235 // Compute upper and lower such that any decimal number
236 // between upper and lower (possibly inclusive)
237 // will round to the original floating point number.
238
239 // We may see at once that the number is already shortest.
240 //
241 // Suppose d is not denormal, so that 2^exp <= d < 10^dp.
242 // The closest shorter number is at least 10^(dp-nd) away.
243 // The lower/upper bounds computed below are at distance
244 // at most 2^(exp-mantbits).
245 //
246 // So the number is already shortest if 10^(dp-nd) > 2^(exp-mantbits),
247 // or equivalently log2(10)*(dp-nd) > exp-mantbits.
248 // It is true if 332/100*(dp-nd) >= exp-mantbits (log2(10) > 3.32).
249 minexp := flt.bias + 1 // minimum possible exponent
250 if exp > minexp && 332*(d.dp-d.nd) >= 100*(exp-int(flt.mantbits)) {
251 // The number is already shortest.
252 return
253 }
254
255 // d = mant << (exp - mantbits)
256 // Next highest floating point number is mant+1 << exp-mantbits.
257 // Our upper bound is halfway between, mant*2+1 << exp-mantbits-1.
258 upper := new(decimal)
259 upper.Assign(mant*2 + 1)
260 upper.Shift(exp - int(flt.mantbits) - 1)
261
262 // d = mant << (exp - mantbits)
263 // Next lowest floating point number is mant-1 << exp-mantbits,
264 // unless mant-1 drops the significant bit and exp is not the minimum exp,
265 // in which case the next lowest is mant*2-1 << exp-mantbits-1.
266 // Either way, call it mantlo << explo-mantbits.
267 // Our lower bound is halfway between, mantlo*2+1 << explo-mantbits-1.
268 var mantlo uint64
269 var explo int
270 if mant > 1<<flt.mantbits || exp == minexp {
271 mantlo = mant - 1
272 explo = exp
273 } else {
274 mantlo = mant*2 - 1
275 explo = exp - 1
276 }
277 lower := new(decimal)
278 lower.Assign(mantlo*2 + 1)
279 lower.Shift(explo - int(flt.mantbits) - 1)
280
281 // The upper and lower bounds are possible outputs only if
282 // the original mantissa is even, so that IEEE round-to-even
283 // would round to the original mantissa and not the neighbors.
284 inclusive := mant%2 == 0
285

Callers 1

bigFtoaFunction · 0.70

Calls 5

AssignMethod · 0.45
ShiftMethod · 0.45
RoundMethod · 0.45
RoundDownMethod · 0.45
RoundUpMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…