mouse pushing icons all over

I have a a PC goes crazy after sitting for a while. When the mouse is moved, it looks like the icons are being pushed around on the desktop. I have turned off the screensaver and it still does it. Does anyone have an idea of what may be causing this? How can it be fixed?