1) The IDE port on the old hard drive will be connected to the caddy via the 2" wide thin flat cable that came with it right? Yes, it will have a short IDE cable and a 4 pin power connector for the drive inside the unit. The outside will have a USB plug and a power supply plug.
2) So I return the jumper on the old drive to master instead of slave, but will that interfere with the existing primary SATA drive in the new pc? Yes, set it to master since it will now be a standalone drive, and no, it won't interfere with SATA because SATA is a different bus technology and has nothing to do with IDE
3) Then I just connect the caddy to the new pc via a USB cable and it comes up as new hardware and I can access its contents, but how does the old hard drive get power to spin? Yes, plug-n-play will detect the new drive and in terms of power, the drive caddy will most likely come with its own external power supply cube. I doubt the USB cable has enough power to power the drive on its own. Think of the IDE drive now as a bulky external USB thumb drive.
It's all pretty simple to grasp if you think about how it all works.