Linux kernel has modular drivers
Compiled into the kernel is the ideal but kernel mods are the norm. Your module plugs into the kernel. Thinks like Virtualbox install there own drivers without a kernel reboot even.
I'd go with your suggestion to try something more focused on hardware support. Mandriva or PCLinuxOS may be a better fit for the user. Mint also.