Are Patch Panels critical ?

Hi,
My cable-men state having a patch-panel is very important in an network cat5(e). They can’t explain why though. I ask them why not just a hub where the wires of the outles end and then get a patchcable from the hub to the server. DON’T DO THIS, they say. But why?
Silence.
How about you people at TechRepublic: WHY (not).
TIA George