J Butch,

How many ports do you have available on the server? Did you limit the number of available ports, which essentially limits the number of concurrent connections. If you only have so many connections, your session may not be ending properly, and the server keeps it open, which is causing the problem. Also, try changing the server from a relay agent, and statically assign the VPN addresses, then see if it works, you may have a relay issue.