Sudden reboot

First, verify that the memory purchased matches the specs for the motherboard. (Check your motherboard or PC manufacturer documentation.) Also, verify that the new memory matches the type and speed of what's already installed. Most newer computers will accept different types of memory, but it's best to match what's already installed.

Try removing the two 256 sticks and boot with the 512 stick only (move it to slot 0 before booting) to see if the problem continues. If it does, you probably have a defective stick - it's happened to me in the past.

If the issue does not continue with the single stick, try leaving the 512 stick in slot 0 and adding the two 256 sticks in slots 1 and 2 and test again.

If the problem recurs when you re-add the original sticks, you could have mismatched memory, or there may be another issue. (See the next paragraph.)

If you can't get the computer to stop rebooting under any scenario, you've got other problems, and more information is required. Specifically when, and under what conditions, does the computer reboot? Any error message or BSOD? If so, what does the message say?

it could be incompatable memory cards. check the motherboard spects for exactly what it wants. and use only that. DDR vs. DDR2, or the speed, or even voltage req. can make a difference.

Some main boards need to have two of the same kind of memory installed. Slot 1 and 3 need to be installed with the same memory, if you add memory be sure it is the same type. Slot one and three are PC2100 DDR 3300 be sure you install PC2100 at least 3300. You can install a lower memory stick but it will lower the higher memory to accept the new memory.

There's several things you'll need to check & do.
Since you don't mention the MB your using it limits the help anyone can provide.
Are you using SDRAM, DDR, DDR2 or RAMBUS which I'll assume your not using.

Do you have the correct memory for your PC?
Are you installing them in the correct dimms?
Both can be determined via your motherboard manual.

Before you install the RAM enter the bios during boot & reset to defaults, this is a often overlooked area when upgrading RAM. Write down the current settings before resetting, I simple make notes in the MB manual beside each setting listed in the bios section of the manual.

Another thing I would suggest is download a little utility called CPU-Z, just Google for it. This app will tell you your memory timings, both the SPD timings (the RAM's real timings) & what their currently set at in the bios.

Seems like all the memory things have been mentioned. There is one basic thing to look out for, make sure that in installing memory you did not bump any cable that may now be not properly seated. Reseat everthing.