Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> [if] you want to retain as much precision as possible and still use floats, don't store it in a float with range [0.0,100.0]. Store it with the range [0.0,1.0].

I just tested this out and it doesn't seem true.

The two storing methods seem similarly precise over most of the range of fractions [0,1], sometimes one gives lower spacing, sometimes the other. For instance, for fractions from 0.5 to 0.638 we get smaller spacing if using [0,100], but for 0.638 to 1 the spacing is smaller if storing in [0,1].

For very small fractions (< 1e-38), it also seems more accurate to store in the range [0,100] since you are representing smaller numbers with the same bit pattern. That is, because the smallest nonzero positive float32 is 1.40129846e-45, so if you store as a float32 range [0,1] that's the smallest possible representable fraction, but if you're storing as a float in range[0,100], that actually represents a fraction 1.40129846e-47, which is smaller.

For the general result, see for yourself in python/numpy:

    x = np.linspace(0,1,10000)
    plt.plot(x, np.float64(np.spacing(np.float32(x*100)))/100)  # plot spacing stored as [0,100]
    plt.plot(x, np.float64(np.spacing(np.float32(x))))  # plot spacing stored as [0,1]


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: