Some domains cannot send emails to our server

Need some expert opinion on this scenario,

I've setup a primary and secondary MX. The primary MX points to the mail server located in our office while the secondary is located remotely. I've noticed that some domain have their emails delivered to our secondary MX instead of the primary.

Majority of incoming emails are delivered to the primary with no problems whatsoever.

The primary email server is always online, and it always shows up on DNS test utilities.