Vista Client can’t connect to VPN Server

Hi! Guys, I have 2 Public IP given by 2 different ISP and I was able to setup a VPN Server using the first public IP address of my ISP, A Windows 2003 server setup as a domain with 1 NIC, I am able to connect to the VPN using the private IP of my VPN server but when I tried to connect from outside of my lan or from my other DSL connection and use the VPN server IP address I was unable to connect to my VPN server.