As an invitee, I look at what the meeting is about, and if I don't think I can add anything, I decline the meeting request with an explanation. I sometimes will call the inviter first to ask for more info about the meeting and then have a short discussion about whether I really need to be there. Rarely do I need to be at a meeting after one of these discussions.

An invite is usually not a command performance, and when it is, the inviter has a responsibilty to note whose presence is required versus those who are optional.