Yes.
It would definitely be better if health care professionals were unable to get to your medical records without your explicit permission every single time, wouldn't it? What if you are unconscious?
Do you have any concept of how many people MUST touch your records for a normal office visit to take place? Would you prefer that there were NO rules to govern what or how those people handle your information?
The county health department cannot request your records without reason, this is part of the protection that HIPAA provides. Public health concerns ARE a valid reason. How do you think things like TB outbreaks are noted and contained so quickly? Records are pulled for everyone who may have had contact. Have you had your immunizations? Are they up to date? Good, you won't be bothered. If you have not, you will be asked to seek medical attention so, you know, you don't die.
What happens if you are in a car accident in another state? Should health professionals be required to seek your permission before viewing your records and providing treatment?
I have news for you. In most cases, your records simply represent a unit of work for the people involved, including the doctor. The sooner they can be shot of your lab-work and your records, the better.
Your doctor CAN, indeed, discuss your wife's medical conditions, care, and payments with you as long as she does not object. However, your doctor WILL NOT do so over the phone simply because you say you are her husband. Were he to do so, the incident would be seen as a breach and reported.