Exchange 5.5 Internet e-mail service

It seems like several times a day my internet e-mail will stop coming in. When I check the service it is still running and looks like everything is ok. I send a test message from my Yahoo account and it does not come in. I do not lose outbound, just messages coming in. When I restart the Internet e-mail service the mail starts coming in again.