As a network grows and changes, different servers begin to take on different tasks. It is usually better to limit the tasks of each server as the network expands, rather than follow the natural course of adding services. In this installment of From the Trenches, we’ll shadow TechRepublic’s IT director, Troy Atwood, and network administrator Mike Laun as they work to optimize their server infrastructure. In this case, they use a third-party software tool called UPromote to demote an overworked Windows NT 4.0 domain controller to a member server.
Get insights From the Trenches
You can learn quite a bit by reading about the methods other administrators and engineers use to resolve challenging technology issues. Our hope is that this column will provide you with unique solutions and valuable techniques that can help you become a better IT professional. If you have an experience that would be a good candidate for a future From the Trenches column, please e-mail us. All administrators and their companies remain anonymous in this column so that no sensitive company or network information is revealed.
The original configuration
When TechRepublic was a much smaller company, its internal network consisted of only a few servers that did multiple tasks. There were two Windows NT 4.0 Servers acting as domain controllers, CorpNT1 and CorpNT2. CorpNT1 did double duty as the company’s Exchange mail server, and CorpNT2 inherited things like WINS, print serving, file serving, DHCP, and IIS.
Atwood had his job cut out for him. “When I got here, I knew we had to get more hardware and break this stuff up,” he said. “We were having all sorts of problems with those servers.”
Atwood describes himself as a big fan of taking network-intensive functions like domain authentication, WINS, and DHCP and isolating them on their own servers. So he began the process of weaning everything he could off CorpNT2 until he got it to the point where it was only a domain controller and a file server.
He also wanted to make sure that he had two machines that would back each other up, one as the primary domain controller (PDC) and the other as the backup domain controller (BDC). If one fails, the other can be promoted to PDC.
Atwood wanted to demote CorpNT2 out of the domain controller role and make it a member server, but because of the functions it handled, he wasn’t ready to rebuild the server yet. Unfortunately, with Windows NT Server 4.0 you can’t simply demote a server from the role of a domain controller to a member server. The only way to accomplish this is by reinstalling NT Server. Thus, Atwood decided to use UPromote to demote the server without having to reinstall NT.
Don’t do this without a safety net!
Demoting a server is a dangerous task and should be approached with the idea that everything on that server could be lost. Back up your data and document all of your directory and subdirectory permissions before taking on this task.
Even testing doesn’t guarantee success
Before attempting to demote CorpNT2 to member status, the IT staff tested the UPromote software on CorpNT1, which was no longer functioning as the Exchange server. CorpNT1 was slated for a reinstall in order to change its role, so it was perfect for testing.
According to Atwood, the test went smoothly. “UPromote worked really well, and it demoted CorpNT1 to a member server. We didn’t have any problems.”
One of the things the program does during the demotion process is take the security database on the server and convert all the user accounts from domain accounts to local accounts. That was fine, but Atwood didn’t expect what happened when his staff performed the demotion of CorpNT2.
“What it also did on CorpNT2 was whack all the permissions on all the file shares. At that time, we had more than 80 gigs of files and I don’t know how many directories and subdirectories,” Atwood said. “It took days to figure that out. That’s the one thing we neglected to test.”
The trip for the security ID numbers associated with the directories and their permissions is one-way, Laun said. Once those had been moved to the local permissions, there was nothing they could do to recover the domain permissions. Trying to do so might have lost the permission information that did survive the transfer. At least they knew who was supposed to have permissions to the directories.
“We had to go into every directory and look at it and change the file permissions from CorpNT2 to [the domain name], and we had to do that for every directory. We had to do that for thousands of directories,” Atwood said.
Laun said one good thing to come out of this mess was that user permissions were set up as groups of common users, which hadn’t been done because the company was growing so quickly that individual users were added to permissions lists with no thought of grouping them.
“Once you have converted [the domain controller], you have to go through all the local SAMs and delete all the accounts out of there, because if you have the same account in the domain controller, and you change the password in the domain controller for the user account, it will no longer match the password in the local domain and you will have a conflict,” Atwood said.
After figuring out and fixing all the permissions on the demoted CorpNT2, the plan was to migrate the server over to Windows 2000 Server. After Atwood acquired hardware with an equivalent amount of storage, he:
- Used RoboCopy from the Windows 2000 Resource Kit to maintain the continuity of the CorpNT2 directory structure on another machine.
- Froze out all activity on CorpNT2 and locked out all users.
- Got the new server ready with the base OS.
- Moved the external RAID set over to the new server and reformatted the RAID set.
- Used RoboCopy again and copied all the CorpNT2 files and permissions to the new server.
“We kept the old CorpNT2 intact in case we had trouble with the upgrade so we could move the RAID controller back over and restore from tape and be right back where we started from,” he said.
If he had it all to do over again, Atwood said he probably would have shut CorpNT2 down, disconnected the RAID array controller, booted back up, locked CorpNT2, demoted it, cleaned up the SAMs, stopped it again, and then reconnected the RAID. He thinks the permissions for all the directories would have stayed the same because they never would have been touched.
He isn’t going to do it over again to find out.
And he cautioned, “Be careful using UPromote. If you’re going to use it on a big file server, be careful because it’s going to whack all your file permissions. You better have them well documented.”
Have you tried a server demotion?
Demoting a domain controller is not a clean and easy task, even if all goes well. Have you ever had to demote a server? Did it have a large database of user permissions associated with it? How did it go for you? Send us a note or post a comment in the discussion below.