Cable modem & Router won’t play together

Hi
I have a cable modem & a D-link DI 604 broadband router. I have 2 XP machines that will not go out to the internet. They are receiving
an IP address’s from the router and I can ping the gateway (router).I have the XP firewall disabled and when I connect either XP machine to the cable modem directly they work fine.

Can someone help I don’t want to pay the cable company for another IP address.

Thanks