Let me Sherlock this:
1. Unless you are different from most IT people, when talking to users you give a short - maybe smartass - answer.
2. If the person that was asking you was above you but neither aware of or could interpret your job description, then you may of liked to be more "complete" in your answer.
I'm guessing number 2 was the situation.
Here's an attempt at an objective answer...
You are a member of a separate culture when you do what you do in a non-IT industry.
Consider this:
The terms that are key and informative that are used by you, are at most, tokens to them. And probably reverse is true for you - though sometimes the impacts may be equal - HIPPA for example.
Being in a non-IT business means that where you are seen in the office, who you are talking to, what meetings you are invited to, all are impacting how these non-technical people look upon your personal worth to the organization.
What I'm leading up to is that maybe you need to know what everybody else does, then you can best address what you do from their viewpoint. They may stick you out on a lone branch in the org chart, but at some time a person like you is dealing with the left and right and up and down of said chart.
See if you can localize and specify the departments and their main actors while answering this question - you did this "where", you worked with "who", these people can do "this job" because you supported them.
Going back to what I referred to earlier, getting this accross is not just a lack of knowledge on their part, it is also a cultural problem. Try to meet them halfway...
Good Luck