Interesting, indeed. All the time we have suspected the CPU and the PSU, but it could be the graphics card that just overheats and shutting off the computer.
I don't know how familiar you are with this stuff. If you are unsure, ask someone for help. I hope my little guide here helps though...
You can find the graphics card if you follow where the cable which is connecting the computer with the monitor is being connected. Then open the case, look on which card the cable is plugged in... and you have found the graphics card. Remove the cable, remove the screw that holds the card in place. Before you remove the card, please touch a radiator first, to be sure to, uh... unload the static electricity charge from your body. Unless a simple touch can destroy the hardware. If there is still a cable connected to the card which goes to the PSU, remove it. Now, remove the card carefully and place it at a safe place where no electricity can harm it (wooden table...).
Take the other graphics card, which should be of the same slot type (PCIe 2.0 or AGP) and plug it into the same slot where the other card has been in. Make sure that it is plugged in steady (I hope that is the right word). Secure the card with the screw, plug in the 6 pin cable, if the card has a slot for it, close the case and connect it with the monitor. When the computer is booting, Windows wants to install the drivers to the card.
If the graphics card is the problem there are ways to get around it. There are so-called Slot Fans that can be placed near the graphics card to support the heat removal.