How to configure A Network with Two ISP

I want some users to get internet from ISP A and others from ISP B but both use the SMTP services. Only a few users and the mail server will use isp A but the rest of users i want them to use ISP B. have one Linksys RV082 router, two switch and two modems supplied by the different ISPs