Hi,
I think with just 4 users it should be fine to use a normal PC
Make sure you are using Core 2 Duo but nowadays you might get a Quad processor in a normal PC. Also do make sure there is adequate RAM for all the user.
Do also keep some space for future expansion.
The most important thing DATA make sure you have backup copy of your data all the time.
Do not over-clog your PC with Mirroring but if you are using a Quad core it should be fine.

Hope this helps you