Protect Windows 2003 from network neighborhood

This might be a dumb question, but here goes:

I would like to restrict my 2003 server from access via the network neighborhood, i.e.: if someone is just browsing along in the neighborhood – when the double click on our server I’d like them to get an access denied msg. How do I do this?

Thanks
Chris