As you describe the problem it sounds like some problem with memory on the card.
It is possible to has defective cells on given address and when you run high end game it will use all video memory and trigger the problem.
If you can try to obtain another video card
from seller just to try.
Another thing - Win2k is not good for games, try to install a Win98 on separate partition and see how it will be with it.
Also try to find old drivers for try, these that come with the card for example.
At last - see that everything in Bios Setup is setup correctly - Memory timing, processor rate, voltage.
Also this can be result if you have overclocked your processor.