Intel Underestimates Error Bounds by 1.3 quintillion
October 10, 2014 4:44 AM Subscribe
Intel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sin) has a maximum error, in round-to-nearest mode, of one unit in the last place. This is not true. It’s not even close.
This thread has been archived and is closed to new comments