To paraphrase Han Solo in "Star Wars," I've worked in IT over 20 years and have seen a lot of strange stuff. I've seen Exchange services fail because someone (other than me) thought it would be a good idea to delete Microsoft .NET 2.0 files rather than uninstall them. I've seen CRT monitors tossed into a dumpster which bounced twice and landed with nary a scratch. I've seen fried systems mysteriously come back to life. I've seen plain-text scripts that refused to work right even though the same commands typed directly into the computer operated fine. I'm currently looking at a strange issue with Active Directory DNS records abruptly disappearing even without DNS scavenging turned on.
In short, I've seen things that aren't supposed to be possible, but yet still happen. I know there's always a technological explanation behind it all (or there's supposed to be one) but due to the shortage of hours in the day I've to chalk some weird oddities up to Walter Cronkite's famous slogan ("That's the way it is,") a few times.
I want to talk about one such oddity today since it seemed a bit too significant and strange to just brush off with an "Oh, well - I'll think about that when I get some downtime." In fact, it's something anyone who works on or runs an IT department should be aware of.
I connect to my company's on-premises Exchange server via Activesync on my Samsung Galaxy S3. My Active Directory password (which changes periodically) controls access to my email.
The other week I changed my password while at the office, then thought nothing further of it. My Droid continued working just fine, accessing email without a hitch. It wasn't until the next day - 30 or so hours after the password change - that I was finally prompted by my phone to enter the updated password.
I had seen this before but it wasn't much of a concern until I reflected that this could pose a significant security risk for companies. While this password refresh delay might be fine for current employees who don't care if they have to update their stored password on their phones today or tomorrow, think what would happen in the case of former employees. A terminated employee - an angry one at that - might have undue access to the company's email system for over a full day. In this era of Bring Your Own Device (BYOD) policies it can be difficult for harried IT administrators to ensure that departing employees disconnect their phone's email accounts from the company mail server and HR may not always know that this should be a required step.
Why is this happening?
First I researched how and why this could be happening. After all, the connection should be straightforward, right?
Note: a cloud-based situation would appear in a similar fashion, with access to the off-premises server(s) via the internet and a firewall between them and the smartphone.
Logically speaking, it seems as though the smartphone should pass through the firewall to the Exchange Client Access server, authenticate with the user's Active Directory account, and then access the email on the Exchange Mailbox server. So, if the password is changed in Active Directory, wouldn't it make sense that the smartphone would immediately be cut off until the password was updated there as well?
Well, it's a bit more complicated than that. When the smartphone authenticates to Exchange it is issued what is called a user token. This token can be used for subsequent access - for a limited time - without the device having to present the password every single time the phone goes to collect email. Think of it as a water park wristband which allows you to go on soaking wet slides without having to constantly show the attendants your ticket. So, it really works more like this:
Note that "good for future access" part. It seems that in some cases the user token permits authentication for quite some time even after the password has changed in Active Directory. The key here seems to be that my phone is using direct push to get new email as soon as possible. This is reported to also happen with disabled Active Directory accounts as well. In both cases, however, it seems that if the smartphone is restarted it then prompts for the new password the next time it tries to collect email. So, clearly that user token is only good for a certain time and must be refreshed at the next phone reboot.
What does the vendor have to say about it?
An article from Microsoft claims that "for performance reasons, user tokens are cached by IIS and updated at 15 minute intervals." IIS is the web service which permits access to Exchange email. Microsoft states this interval can be adjusted in the registry to set it for lower than 15 minutes and that restarting all IIS services (via the command iisreset) will refresh the token cache (this information also appears in a separate article).
This doesn't seem to be the correct answer since my password was good for well over a day - much longer than 15 minutes, obviously - but it represents the grain of salt I've become accustomed to taking while reading Microsoft support articles, which in my experience are infamous for telling you that such-and-such problem was fixed by a service pack t hat's been on the problem system in question for years. Better answers are generally out there, courtesy of Google (though you do run into your share of red herrings, such as the ever-popular "I think that's a DNS issue" which seems to be the cause of everything from male pattern baldness to UFO abductions, at least in some circles).
I did a bit more research and found that while - yes, an IIS reset will fix this, it's not feasible to go resetting critical services every time an employee is terminated since that can have a nasty tendency to impact other employees. It's also possible to achieve the same end by resetting the Application Pool used by ActiveSync (At the Exchange Client Access Server, click Start, click Administrative Tools, click Internet Information Services (IIS) Manager, then expand the server name, click Application Pools, right click the MSExchangeSyncAppPool and click Recycle). Furthermore, moving the terminated employee's mailbox to another Exchange database will also correct this, but all of these are a bit more complex than they need to be.
What's the best thing to do with terminated employees?
You should have a written BYOD policy or Mobile Device Policy (templates provided on the Tech Pro Research site can be modified to meet your needs) coordinated through HR and the IT department so that upon employee termination any company-owned devices are turned in and employee-owned devices are inspected to ensure all company accounts, data and other information have been removed. The devices should then be disassociated with the user account and Activesync turned off for that account. Your policy or policies should also include a tenet that these devices can be remotely wiped at the discretion of the IT department if it is believed the above steps have not fully been performed.
Whether this is the case or you simply need to remove the mobile phone partnership between the device and the user's account, make sure to follow the appropriate steps. For instance, in Exchange 2010 you would access the Microsoft Exchange Management Console, expand Recipient Confirmation, choose Mailbox, find and click the user, then click "Manage Mobile Phone" in the set of options to the right.
Choose "Remove mobile phone partnership" or "Perform a remote wipe to clear mobile phone data" depending on your needs then click "Remove."
Now go to the user account properties in Exchange (access the Microsoft Exchange Management Console, expand Recipient Confirmation, choose Mailbox, find and right click the user and choose Properties, then select the Mailbox Features tab) and disable Activesync. Disable Outlook Web App while you're there, too:
This is a good example of something which was designed to make technology work easier and more efficiently but yet could also turn out to be a problem if not managed properly. The fact the user token permits access after the password has been changed is preferable to experiencing user account lockouts for too many failed logon attempts, but can have more frightening implications. Good policies and user/device management are the key to overcoming this security risk, and don't forget to keep a healthy communication between HR and IT so you're aware of employee comings and goings!
Scott Matteson is a senior systems administrator and freelance technical writer who also performs consulting work for small organizations. He resides in the Greater Boston area with his wife and three children.