Now might be a good time to have a security audit done!
Hopefully this person has signed an agreement before he departed that specifically addresses the issues you are worried about. It doesn't prevent access but may be a deterent and gives you some legal reassurance.
Check for generic accounts that have dial-in access and remove if possible or change the passwords. As already suggested, change all the admin-type passwords as soon as possible. You should already know which passwords dowhat, so nothing unexpected should break - but keep an eye out for this. Check for unusual scheduled server jobs that might be malicious (eg changing passwords, re-creating backdoors).
Physical security is also important - change keypad PINs, cancel his access cards, ensure any tokens are returned and disabled, etc. How paranoid are you - need to change key locks on security doors etc?
For routers and firewalls, and probably any server too, do a visual inspection and trace any cables that you can't explain. Even worth double-checking the ones you can explain.
Ensure that this person was not a primary or authorised contact for things like off-site tape storage, maintenance agreements, supplier or support companies, secure web sites, certificate authorities, DNS registrations, etc.
It really helps to think paranoid. If it is not in your nature then pick up a book on practical information security. You could seek professional advice from a computer security company.
HTH
William
