If you can ping the address alright than there doesn't seem to be a connectivity issue. But rather a configuration issue.
Check if you can connect to Terminal Server, FTP, VPN.
How is the server connected to the Internet? I assume that it is connected directly to the Internet and has a public IP address, and not NAT/PAT.
If it is set up using either One to One NAT, or PAT:
Make sure that on the test client computer that you are trying to connect to the public IP address is not a computer that's local. Since using NAT and then connecting to a one to one NATed address requires for your local computer to go out to the NAT server then for the NAT server to go back in into the local network. What happens is that on most NAT server it doesn't get routed at all, since the server gets confused on how to route it, because the Source and Destination MAC and/or IP address are the same.
You should test it only from a computer that's connected to the Internet from another source and not localy.
Telnet to the port in question to see if it works. The port numbers are:
Terminal Server = 3389
FTP = 23 or 21 I dont remember (one is the telnet service and the other FTP)
VPN (PPTP) = 1723
if it doesn't make sure that IP filtering is not enabled. If it is enabled open the ports in question.