Advise - Switches connections.

I need advise in the correct configuration of switches across the building. I have the server room in the second floor, and need to connect 20 Pc's at the 4th floor, 5 at 1st floor, and 5 at 5th floor. The main patch panel is at the server room (2nd floor, cat5 cabling), connected to a Dell PowerConnect 5448. I have an uplink cable to a swith at the 4th fl. (gb ports), and a small patch panel to handle the pc's.
My concern is about performance degradation. From the 4th fl. swith, I need to connect another switch, and my doubt is if it is the same if I use another switch or is better idea to wire single cables directly from the second switch.