Problems with IP and DNS in win2k

I have a cliente that has 5 workstations with win2k pro and a win2k server. To access the internet they have a router (model Micronet IP router SP866) I have configure the router well. On each workstation I have done the following configurion of theTCP/IP protocol: Gateway, I put the router IP address; Dns, I put the primary and the secondary IP address of the internet service provider. The problem is that each time the computers is switched on or accessing other computers in the network, it activates the router, making a connection to the internet, which I don’t want. I want the router to make the connection to the internet only when I use Microsoft Internet Explorer or Microsoft Outlook.
Is there some configuration that I missed doing?
Please do help me. It is most urgent. Thank you.