Problem accessing site by IP Address

We have an intranet site on an IIS server, which is set up to automatically detect a user’s NT ID and display results accordingly. When the site is accessed through IE by using its host name, it works OK. However, if you try to access it by using the server’s IP address it brings up the NT challenge login box (then lets you in once details are entered). DNS resolves the host name to the same IP address, so why the difference?