remote desktop through cisco pixfirewall

I have a win2003 server behind cisco pix 515E and I certainly permit RDP port 3389. When I use remote desktop to connect the 2003 server through firewall and open the IE browser, the remote desktop always gets disconnected after I browse several web pages. But if I remove the pix firewall, the remote connection works fine. Is there any additional setting for pix I missed besides permit RDP port 3389? or probably I miss-configure the pix?