If you look, you'll see I only talked about free/open, not mentioning "open source".

Whether he (or anyone else) likes it or not, "open source" is the popular phrase that most people use when they mean "free (as in speech) and open" and that's not likely to change. In the same way that it's not likely people will stop saying Linux when they mean GNU/Linux, or stop saying "Hoover" when they mean vacuum cleaner, or any of a miriad of similar common terms.
The fact that open source is also used to mean things that don't meet the definition of free and open is another problem.

Personally I think it's important not to get too hung up on it. While I can see where RMS is coming from, I do think his hard line attitude does put a lot of people off and they fail to hear the underlying message.