MCPcopy Index your code
hub / github.com/pquerna/ffjson / adjustLastDigitFixed

Function adjustLastDigitFixed

fflib/v1/extfloat.go:496–526  ·  view source on GitHub ↗

adjustLastDigitFixed assumes d contains the representation of the integral part of some number, whose fractional part is num / (den << shift). The numerator num is only known up to an uncertainty of size ε, assumed to be less than (den << shift)/2. It will increase the last digit by one to account

(d *decimalSlice, num, den uint64, shift uint, ε uint64)

Source from the content-addressed store, hash-verified

494// when the fractional part is greater than 1/2, and will return false if ε is such
495// that no correct answer can be given.
496func adjustLastDigitFixed(d *decimalSlice, num, den uint64, shift uint, ε uint64) bool {
497 if num > den<<shift {
498 panic("strconv: num > den<<shift in adjustLastDigitFixed")
499 }
500 if 2*ε > den<<shift {
501 panic("strconv: ε > (den<<shift)/2")
502 }
503 if 2*(num+ε) < den<<shift {
504 return true
505 }
506 if 2*(num-ε) > den<<shift {
507 // increment d by 1.
508 i := d.nd - 1
509 for ; i >= 0; i-- {
510 if d.d[i] == '9' {
511 d.nd--
512 } else {
513 break
514 }
515 }
516 if i < 0 {
517 d.d[0] = '1'
518 d.nd = 1
519 d.dp++
520 } else {
521 d.d[i]++
522 }
523 return true
524 }
525 return false
526}
527
528// ShortestDecimal stores in d the shortest decimal representation of f
529// which belongs to the open interval (lower, upper), where f is supposed

Callers 1

FixedDecimalMethod · 0.70

Calls

no outgoing calls

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…