Just downloading the file doesn't do anything. You need to execute it.
I suppose if you've got windows configured to autorun, you save it to a flash drive, remove the drive & reinsert it, then you could be automagically infected.

Then again, I've seen a user open an email "from themself", download an attachment they supposedly sent themself, unzip said attachment, which luckily they included the zip password in their email, and execute the file they had no recollection of.

Just downloading the file is more like loading a bullet into the gun. It has the potential to fire, but it doesn't fire itself, you need to pull the trigger.