POP3 on Exchange 5.5

I’m hosting my own email with Exchange. When in the office everything works fine. I can send and receive all email. When using RAS I can receive email fine, but when I send I get a message “relay denied”. Also, if I am out of the office and I connect to the internet using my ISP I can’t get my email from my server. My POP3 server is correct. My proxy server is relaying correctly on port 110, but my server doesn’t seem to be running POP3 correctly even though it is enabled. When I try to telnet to my server’s IP port 110 it will not connect. But if I try to telnet to 127.0.0.1 110 it says POP3 is ready. Do I need to somehow bind POP3 to my servers IP address?