problem with email at home

i use outlook express and windows xp. i can send email no problems. anyone that sends me email gets msgs that its undeliverable or delayed.i do howeever get their email it tells them i didnt. my isp tells me its due to norton anitvirus and turn off email scan. if i send myself a test msg ie email myself the email comes right to me then for next weeek or so you get all types of delayed or undeliverable msgs. this is for a home computor