64-bit slower than 32-bit with same amount of memory?

I suppose having a 64-bit system without correspondingly more memory installed would not help with the efficiency. In fact it could have adverse effect on performance. Am I right?

Say, for example, a system with 2G memory if installed with a 32-bit Windows 7 system could perform better than with a 64-bit Windows 7 system installed.

To take advantage of the 64-bit system, I understand we have to install more memory. Analogically, it is just like a big car needing more petrol for running the same speed.

Would you think so?