Safe mode drivers

I have thought for a long time now that the device manager should appear the same in both normal and safe modes, so when I upgrade a machine’s motherboard I clean out the device manager in safe mode and let windows find what it’s really going to use. My queston is can drivers that are visable in safe mode cause system instability in normal mode?