4 Comments
The post below has been deleted. Click to close.
This post has been deleted. Click here to see post.
by
Osud, Dec 9, 2008, 3:29 AM
- Report
The post below has been deleted. Click to close.
This post has been deleted. Click here to see post.
It looks like an odd interpretation of order of operations- that's clearly being read as (-2)^4.
Back in college, one assignment in a numerical analysis class I had was to run a calculation through various calculation engines to detect roundoff error. One of the systems I used was an old (Mac OS 9) version of Excel, and I got a very odd result. When I presented this to the professor (William Kahan, a leading expert on this sort of thing), his reaction was something like "They shouldn't be doing that!".
The specific issue was that certain subtraction results close to zero were rounded to exactly zero, at a resolution slightly coarser than floating point numbers away from zero.
Back in college, one assignment in a numerical analysis class I had was to run a calculation through various calculation engines to detect roundoff error. One of the systems I used was an old (Mac OS 9) version of Excel, and I got a very odd result. When I presented this to the professor (William Kahan, a leading expert on this sort of thing), his reaction was something like "They shouldn't be doing that!".
The specific issue was that certain subtraction results close to zero were rounded to exactly zero, at a resolution slightly coarser than floating point numbers away from zero.
The post below has been deleted. Click to close.
This post has been deleted. Click here to see post.
This has driven me crazy for a long time. It's really poor. I run into it a lot when I'm writing a gaussian:
= exp(-x^2/2/sigma^2)
I've had to make it a habit to write the astonishingly ugly
= exp((-1)*x^2/2/sigma^2)
Agreed that it's very weird that Excel does this.
= exp(-x^2/2/sigma^2)
I've had to make it a habit to write the astonishingly ugly
= exp((-1)*x^2/2/sigma^2)
Agreed that it's very weird that Excel does this.
The post below has been deleted. Click to close.
This post has been deleted. Click here to see post.
Archives































































Tags
About Owner
- Posts: 16194
- Joined: Mar 28, 2003
Blog Stats
- Blog created: Jan 28, 2005
- Total entries: 940
- Total visits: 3309455
- Total comments: 3879
Search Blog