Screen-Saver is a misnomer; a carryover from the age of the monochrome dinosaurs and unnecessary from a technical standpoint. They also use resources to monitor the system. The only benefit is to hide sensitive data when one is distracted and prove to others that the machine is not being utilized to its fullest potential. They also consume 15-20% of system resources, and interfere with background processes. Now that my personal prejudice is out of the way? And, since you insist, watch for the "teapots" in 3D-Pipes. You're going to be staring at the screen for one minute waiting for it to kick-in, what's another four to see the Easter Egg.
Only two events should cause, or rather inhibit, the animations: Keyboard or Mouse activity. Some ill-behaved background tasks simulate both. Virus monitors, FTP sending a ^G (bell,) Kernel monitors, etc. Since you are not providing keyboard activity the mouse becomes suspect. Any vibration will reset the timer and prevent the activation of the "screen-saver." By way of allegory, my CD-Burner ceased to make coasters once I moved from atop my tower.
Intelligent "mice" may also send "keep-alive" signals, thus resetting the timer.
More likely is something in the power profile sending an "Are You Alive?" to the processor thus waking the "sleep-mode" that has been set.
Bob