usb problems

The strangest thing is happening:

My windows xp pro machine will only boot with a USB drive in the USB
port.

If I take the usb drive out and boot this happens:

first the windows xp screen comes up, then after a few seconds it will
go to a black screen and do nothing.

I have to hold the off button for 30 seconds to shut it down.

then I put the usb drive in, boot up and windows boots up with no
problems.

I have checked bios settings, bios set to boot only to the SATA HD
and shut off booting from USB.

The only strange file on my usb drive is a BOOTEX.log file.