Precision of Real Data Types in C# - TechRepublic
General discussion
September 26, 2002 at 01:27 PM
herong_yang

Precision of Real Data Types in C#

by herong_yang . Updated 23 years, 9 months ago

C# program:

using System;
class Precision {
public static void Main() {
float f;
double d;
decimal m;
for (int i=1; i<=2; i++) { f = (float)i/3; d = (double)i/3; m = (decimal)i/3; Console.WriteLine("Testing {0}/3:", i); Console.WriteLine(" f = {0}", f); Console.WriteLine(" d = {0}", d); Console.WriteLine(" m = {0}", m); Console.WriteLine(" f*3 = {0}", f*3); Console.WriteLine(" d*3 = {0}", d*3); Console.WriteLine(" m*3 = {0}", m*3); Console.WriteLine(" (double)f*3 = {0}", (double)f*3); Console.WriteLine(" (decimal)f*3 = {0}", (decimal)f*3); Console.WriteLine(" (decimal)d*3 = {0}", (decimal)d*3); Console.WriteLine(" (double)((float)i/3)*3 = {0}", (double)((float)i/3)*3); } } } Output: Testing 1/3: f = 0.3333333 d = 0.333333333333333 m = 0.3333333333333333333333333333 f*3 = 1 d*3 = 1 m*3 = 0.9999999999999999999999999999 (double)f*3 = 1.00000002980232 (decimal)f*3 = 0.9999999 (decimal)d*3 = 0.999999999999999 (double)((float)i/3)*3 = 1 Testing 2/3: f = 0.6666667 d = 0.666666666666667 m = 0.6666666666666666666666666667 f*3 = 2 d*3 = 2 m*3 = 2.0000000000000000000000000001 (double)f*3 = 2.00000005960464 (decimal)f*3 = 2.0000001 (decimal)d*3 = 2.000000000000001 (double)((float)i/3)*3 = 2 Question 1: Why ((float)1/3)*3 = 1? Question 2: If ((float)1/3)*3 = 1, then why ((decimal)1/3)*3 = 0.9999999999999999999999999999? Question 3: If ((float)1/3)*3 = 1, and ((decimal)1/3)*3 < 1, then is the float type more accurate that the decimal type? Question 4: If f is (float)1/3 = 0.3333333, then why (double)f*3 = 1.00000002980232? Question 5: If (double)f*3 = 1.00000002980232, why (double)((float)1/3)*3 = 1? Thanks. Herong Yang

This discussion is locked

All Comments