You seriously need to upgrade your service pack. Service Pack 5 is the most stable to date. Don't bother with 6 for a while because it is full of bugs and most vendors still don't support it.
Service pack 3 isn't even Y2K compliant, so there's no telling what the problem could be. I'd be willing to bet that your response time would improve it if updated.
Also, you may want to set up Network Monitor to analyze the traffic. There could be a problem there as well. Also, use Performance Monitor to see if the sytem is maxed in it's use of resources. These two things will help you figure out what the bottleneck is.
Good Luck!