Unfortunately, installing and uninstalling different programs, system and security updates piling upon each other, different add-ons to different parts of your system that you may not even have noticed, viruses and trojans, which sometimes are not completely deleted...
All these things build up over time.
Some people I know from time to time make a backup of all their data, then they do a clean installation of the system with needed updates on a freshly formatted disk, in order to get rid of such garbage.
I haven't done this myself, but might consider it if I feel my system is becoming unstable and slow. So this is just as food for thought.