Deleting Files

I would like to know how to totally delete a File and why does a file/folder is still present on the disk after deleting either from the desktop or control panel? I am also wondering why a program is still on the Add/Remove Program window after removing/deleting it. Can someone enligthen me? I’m getting confused. Note that I have emtied the recycle bin and restarted my computer.