These are at the top of my list.

With Point 3, in my experience cooling is most important for Hard Drive and Video Card (if you go accelerated), not so much CPU. However, the cooler you keep a modern CPU, the more speed you will get due to the fact the CPU's are built to slow down when overheated. The first Server I built had cooling fans everywhere due to a very hot RAID array it housed. When I left the job, the replacement tech failed to replace one of those fans when it died. Two months later, the Hard Drives started to die. Same with accelerated video cards. These don't last long hot. I5 and I7 CPU's also fail early when run hot due to their on die Video Driver - this is a relatively new consideration.

The case: Go for the biggest you can get away with - once again for cooling. However, if you are on a budget, stick to a smaller, more rigid case rather than a cheap big case with heaps of flex. Better to have a smaller, more rigid case with more fans than a big, poorly built case. I have seen many motherboards fail early due to case flex.

Power: This really is the heart of the computer, try not to skimp here or the computer will be unstable not long after build. If you live in an area with bad power delivery (such as the Gold Coast, Australia), invest in an online UPS that is big enough to cope with the computer and the screen. This will further enhance your stability.

Memory: If stability is a big factor, ALWAYS use ECC memory. Better to have 16Gb of ECC than 32Gb of non ECC if stability is key to your build.