enter the bios, check if the second drive is seen correctly. to be sure there should be a "detect IDE channels" or something similar available.
As I guess your second HDD is on the second IDE channel, make sure it is enabled in the bios
look for the CHS numbers on the label of the HDD and make sure that they are what the bios detected. worse case scenario, you would have to get the bios autodetect the HDD config at each boot. Takes a handful of seconds more at each boot but this is usually a very safe settings.
Check the jumpers, make sure it is set to "slave" if there is a master on the same channel, and that the master is actually set to "master" and not "standalone" if the options are different for "standalone" and "master"
as well I encountered in the past a situation were a faster slave prevented the slower master to actually handle the bus. this usually prevented the system to start.
if your second HDD is alone on second IDE channel make sure the jumper are set to "master" or "standalone"
finally if your second HDD is NTFS formatted you should see the drive letter but not be able to open it. Then you'll want to download the NTFS driver for Windows 98 by sysinternal
just google ntfs98 and pick your preferred.
The free version offers only read access. I do not know what happened to the paid (i.e. write) version since sysinternal got acquired by MS.
HTH
Philippe