Default login server with multiple sites

I have a Windows 2003 Domain with two sites and a Domian Controler in both sites. When some users login, they are being loged into the domain against the server in the other site. This causes the login to take upto 10 mins. How do I set which server to log into?