Delivery to recipients has been delayed.

I receive “Delivery to the following recipients has been delayed” from my exchange server whenever I try to email to any Yahoo email address. There are NO errors, just a delay notification and 90% of the time the email never goes through. I can track my sending and it appears to leave my server but it leaves me lost in space as for a reason or what happens that causes this. I have asked Yahoo and they first said it was possibly a glitch on their side but later told me the problem must be mine. I am lost for a solution because no errors or daemons, exccept for the delay notice and that once in a while the email goes through.