Outlook 2003: Prevent attachments going to all recipients

I use Outlook and would like to be able to send attachments to only certain people on an email string. I would like it so that I can allow my colleagues to know that I replied to a specific issue but not bombard their inbox with attachments that are no earthly good to them
Basically, attachments for those on the TO line and no attachments for those on the CC line

Is this possible? if so can anyone help?