Okay, here's what to do.
1) Verify you have a dial tone on that line (plug a handset phone in and listen)
2) Go under your device manager under system in the control panels and delete the modem.
3) Reboot, when it detects the modem, direct the system to the required driver files (on CD, Floppy, wherever).
4) Open the device manager again and see if the modem has a question mark etc to show there is a problem.
5) If it looks okay, close that and go under the modems section in control panels.
6) Select the diagnostics tab and click on the modem.
7) Click more info to test the modem. If all is well, it will list a series of details. If it says the port is already open or some similar error, you most likely have an IRQ conflict with another piece of hardware on your system (a serial mouse or other serial device is the most likely culprit).
If at this point all looks well, check your software settings to dial you ISP...and then try to connect.
If you run into trouble going through these steps, make sure to post the details and we'll help further.
Good luck!