You're running into the limitations of floating point arithmetic. The basic issue is that just as 1/3 cannot be represented in base 10 decimal notation, it can be approximated by .3, .33, .3333, etc. Similarly, it happens that a fraction like 0.1 in base 10 cannot be represented in binary, but it can be approximated. The result of this approximation is what you see in your floating point arithmetic results.
Depending on the level of complexity you're looking for in an explanation, here are some links:
IEEE FP spec
http://www.psc.edu/general/software/packages/ieee/ieee.html
Explanation of FP in Python (but applicable to all languages using FP arithmetic) - well written
http://www.python.org/doc/current/tut/node14.html
Jay