com port conflict

I have a server with 2 built in com ports. I bought an ISA card with two more com ports on it. The first two ports have i/o addresses of 3F8h and 2F8h respectively. I set the jumpers on the ISA card for 3E8h and 2E8H respectively.
I set the IRQ’s (11 and 12)to legacy in the bios setup and when the computer boots, the bios recognizes 4 ports with the above i/o addresses but when linux boots, it assigns irq 3 and 4 to both pairs of com ports and the 2nd pair doesn’t work.
Can anyone help?
Thanks,
Ron