Problem with Terminal Services on Windows Server 2003

I have a client that who is running a Terminal Server on a Window Server 2003 box. The server is up to date and has all current patches.

The problem occurs when the users are logging via TS. They get bumped off from time to time. There are only 5 users that make use of TS and never more than 3 at a time. There are no errors in the logs and there is no problem with the network.

I have been chasing this problem for 2 weeks now and I have run out of ideas. If any one has any ideas please let me know. I will provide additional indformation if it is needed. Thank you.