Yes
Unfortunately, it would seem that some of the files do end up blowing up in size a bit. Basically though, what I have learned to do if such an issue occurs is to launch the rather large PDF file and save it out as an "Optimized PDF". This will allow Adobe Acrobat Reader to analyze the document once again and cut out the fat. Now on some occasions, according to various sources on the web, you can save a secured PDF out as an "Optimized PDF" and that will break the shackles of security free from the document, all the while keeping it lean. However, I have noticed that not all secured documents can be treated in this fashion, so definitely give it a try and let me know what you think.
Also I noticed you had issues with the text quality. That can be easily solved simply by cranking the DPI up to the highest setting just before printing to XPS.
Ultimately, there is more than one way to skin a cat, and I am open to any suggestions anyone has.