After Hours

Ask something
Question Locked

AD clients NTuser.dat locked on login

0 helpful
28 replies

by keith.abbott

Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

Hi,

We're having problems with some of the users on our AD domain. They get userenv 1508 errors (among others) when they try to log in the first time in the morning.

Some background:
Our original domain is an nt domain. All users on the AD domain (except 1 or 2) were migrated from the NT domain to the AD domain using microsoft's ad migration tool.

The AD domain is win2k3 r2 (upgraded to r2), the clients are xp.

We were migrating users from the NT domain in a staged fashion and some had been working fine for a couple of months.

One Friday night I made some GPO changes to try to correct some time sync problems, and get the PDCE to sync with an external source.

The following Monday morning there were a rash of issues with people getting warnings that their profiles couldnt be loaded and temp profiles were being created.

It would be an unbelievable coincidence if the two events were not related (I dont believe it).

Most were able to reboot and get loaded correctly but several profile rebuilds were required.

I backed out the changes I had made previously but the problems persisted in the following days. The user could log in and out all day long with no problem, but overnight the issue was created.

At suggestions from another site, I tried GPOfix to return the group policy back to default (which was fine because previously the default group policy had been modified and this gave us the chance to default it and, instead create our own adjunct policies). However the problem persists.

About 150 of somewhat over 300 users have been migrated. Of the 150, I'd estimate about 50 of those have experienced the problem. Maybe 6 or 8 experience the problem very consistantly. Others come and go. Some seldom have the problem, many have not yet experienced it. We had 11 users reboot yesterday, 16 the day before. The 1 or 2 that were built on the AD domain have not yet had a problem.

Using Sysinternals procmon and process explorer we have determined the cause of the problem (at least in the case of our test subject) is that system (PID 4) is locking the profile's NTuser.dat.

The problem is profile specific as we had our test user attempt to log in and when it failed, log off and attempt to log in under a different account. The attempt was successful.

For those who have daily problems, rebuilding their profiles usually give good results, but not always permanent.

We have tried installing user profile hive cleaner. Again this had some but not universal success. (and I consider it a band-aid). It worked fine for some of our sample group, but the main test user had little or no improvement.

By logging in under another account, and manually loading the users profile we have been able to determine that, in the case of the test user, the event causing the issue occurs betwen 20:00 and 20:45. The user logs out about 15:30 each day but I don't think the event is related to the logout time (I could be wrong).

I think those are the main facts (there are certainly a hundred others).

Does anyone have any idea how we might approach finding a solution to these problems?

Thanks for your help.
k

28 replies

by keith.abbott

I stated above that the event was occurring 20:00 to 20:45. That was true over the period of several days but I noticed last night that it had shifted to sometime between 20:45 and 01:00 the next morning when I tried it again for grins....

by cg it

from NT to your new AD. I'm also thinking of the sync to external source, and not sure what that really means. Are you syncing with your NT domain?

But, my 2 cents is, it's a profile problem and it's the migration from NT to AD.

If the SID and profiles don't migrate right, and NT to AD without an upgrade to AD, that's a problem, your going to get this behavior.

by keith.abbott

On the 'sync to external source' I'm just talking about getting the PDCE's clock to sync to a standards based source like time.nist.gov. That is working well now.

We had considered the idea that the migration was the source of the problems as well - although that would reduce the GPO changes to the status of a trigger or complete coincidence.

But we've deleted the user profile on the machine, then had the user log in - which would create a new AD profile on the user's machine. It seems that would resolve any migrated user profile issues.

What additional steps do you recommend that we could give a shot?

thx,
k

by cg it

how did you migrate the computer accounts? or did you create new computer accounts and join them to the domain?

If this happens to new user accounts, then I would say it's that group policy that's doing it.

run the RSOP tool on the users OU to see what their group policy is if there is a GP on a user OU or do it domain wide.

http://support.microsoft.com/kb/323276

by keith.abbott

We migrated the machines using microsoft ad migration tool, which migrates the machine and the local profiles. I'm not sure how domain accounts were brought over from the NT domain. That was handled by someone else who has since taken a position with another company.

We haven't really had any new accounts that we have built on the domain. We have considered - on the test user - deleting her account from the domain, her domain account from the PC, creating a new account on the domain, then having her log in to the domain and copying her user data (which would be copied into a temp folder) back to her new profile. To see if that would have any effect. I was kind of saving that as a last resort.

I had run RSOP early on in the problem but it might be a good idea to take another look at that as well...

by cg it

in using the ADMT to move machine and local machine user accounts [local machine aren't domain but you can make em domain].

I'd get a user, backup their profile, unjoing the comp from the domain,rejoin it, create an Active Directory user account on the DC. Then let her log in, log off.

I'ld also export her domain profile, do the repair corrupt profile with the exported profile, import it and try that see if her account locks out.

if not, I'd then try this with a computer that hasn't been unjoined from the domain, see if it's the machine computer account that was migrated.

by alashhar

the users SID have not migrated during the migration from NT to 2k3. so the users SID number in the users PCs are not the same in AD. or users? ProfileImagePath registries are duplicated with other users' ProfileImagePath. therefore try the following:
solution 1:
(1)Please make sure you have backed up the registry key before you delete anything.
(2)run regedit and go HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList\
(3)delete the SID keys in the all profiles (ProfileList\).
(4)Test and see if the problem is fixed
solution 2:
http://support.microsoft.com/default.aspx?scid=kb;en-us;811151

by keith.abbott

Hi guys,

thanks for the help so far. I tried Alashhar's suggestion tonight. It essentially has the same effect as rebuilding the profile since windows can't find the profile info in the registry. It created a "user.domain" path in documents and settings. I copied the users files back in from there. We'll see what happens tonight/tomorrow...

I also ran a gpresult /z on the users machine and compared it against my own. I didn't see anything there but I can post it if you want.

CG IT's full-blown removal/readd is still on the table as well...

by cg it

as long as the computer account is ok, then creating a new user profile in AD, then taking the guts of the old user profile and putting it in the new AD profile sans the 3 main files ought to work. but if the computer account isn't correct eg something messed up with ADMT moving them over, then I think your freeze isn't user accountbased but computer accounts [bad SID].

by alashhar

is event viewer show any error? if so please post it

by keith.abbott

So far our test user has gone 3 days without a reboot (although the first one doesn't really count since I was on there mucking, the night before).

We'll see where it goes. If we can get the clients clean, we can continue with the migration.

But that raises another question. Should we skip migration all together, remove the machine from the NT domain manually add it to the AD domain, have the user log in, then copy the stuff from their old profile to their new one?

It's not ideal because a lot of user settings don't come over with that method (but its better than these problems)

What do you think?

k

by cg it

unjoin NT, rejoin AD, redo new AD profile with old NT profile?

by keith.abbott

Actually as I look, a few additional errors have started appearing at some point. In chronological order, the following application errors occur (Note the user logs in around 0630):
0033 userenv 1054
Windows cannot obtain the domain controller name for your computer network. (A socket operation was attempted to an unreachable host. ). Group Policy processing aborted.

0226 userenv 1054
0628 userenv 1508
Windows was unable to load the registry. This is often caused by insufficient memory or insufficient security rights.

DETAIL - The process cannot access the file because it is being used by another process. for C:\Documents and Settings\T00954\ntuser.dat

2 sec later userenv 1502
Windows cannot load the locally stored profile. Possible causes of this error include insufficient security rights or a corrupt local profile. If this problem persists, contact your network administrator.

DETAIL - The process cannot access the file because it is being used by another process.

<1sec later userenv 1515
Windows has backed up this user's profile. Windows will automatically try to use the backed up profile the next time this user logs on.

<1sec later userenv 1511

Windows cannot find the local profile and is logging you on with a temporary profile. Changes you make to this profile will be lost when you log off.

The 1054's are 'new'. Please note that I have tested the following.

Flush the user's dns cache
ping PDCE - resolves fine
Ping DC (secondary DNS) - resolves fine
ping PDCE.Domain.com - resolves fine
ping DC.Domain.com - resolves fine

OK I've just located an issue with DNS resolution - but since it only involves pinging the old NT domain servers, it may be completely unrelated. I'm afraid I'll have to include some background GC has already heard in a different thread:

Our original NT domain had it's own dns at x.x.x.30 . When we created the new AD domain, naturally the first DC had DNS - it was at x.x.x.48.

Later on, we created a new DNS server on the AD domain on a DC - decommissioned the DNS server on the NT domain, and moved the IP address - x.x.x.30 - of the old NT dns server, to the DC. We bound it's DNS to x.x.x.30, and disabled it's old nic so x.x.x.30 became it's only address.

So, and as it stands today, all AD domain users point to x.x.x.48 as the primary dns and x.x.x.30 as the secondary dns. For NT domain users it's just backwards of that.

I set up a packet capture on our DNS servers and had the AD test user attempt to ping the DC from the NT domain. The suffix search order for an AD user is 1.ADdom.com, 2. NTdom.com, 3. RemoteSiteDOM.com.

The remote site point's up to our corp offices: it is also where x.x.x.48 DNS forwards for internet resolution.

Viewing the packet capture, I expected to see a request to resolve NTPDC.ADDOM.COM - which should fail, then NTPDC.NTDOM.COM - which should succeed. Instead, NTPDC.NTDOM.COM failed and the request was made for NTPDC.RemoteSiteDOM.com. It was correctly resloved by x.x.x.48 but with the wrong suffix in the return. I have no idea why.

Further, it doesn't seem like this should come into play at all with the issue at hand (profile locked because ntuser.dat is in use) so it may just be a side issue I need to investigate.

PS, as a side note, I repeated the test of having the test user ping the old ntPDC only this time via fqdn (ntpdc.nddom.com) and x.x.x.48 correctly resolved the name. I don't understand why it didnt when it was an appended suffix!

by keith.abbott

This doesnt make sense. I just found the following statement on support.microsoft.com:

The following methods of distribution are not available for pushing the domain suffix search list to DNS clients:
Dynamic Host Configuration Protocol (DHCP). You cannot configure DHCP to send out a domain suffix search list. This is currently not supported by the Microsoft DHCP server.

This is listed for NT4/win2k. Our DHCP server is Win2K server. Is this correct?

by cg it

which is to say override a static list or a list from options configured in DHCP. Clients that use DHCP to get it's configuration eg DNS servers does so from options configured in DHCP. you can't specify a list to use other than those you list in the options.

The list in options is what the client will use in decending order. If the first DNS server is unavailble it will use the second one. If that isn't available, then the 3rd one. If the workstation can not find a DNS server that can answer the query, which in this case authenticate to a domain controller for the domain, you get that can not contact the DC to authenticate with. If the user has a local machine profile, that is the same account for the domain profile, then that loads, but Group Policy from the domain is aborted and the local machine group policy, if any, is loaded.

by cg it

this one, because of the way you migrated from NT to Active Directory using ADMT, I think is a security permissions problem.

0226 userenv 1054
0628 userenv 1508
Windows was unable to load the registry. This is often caused by insufficient memory or insufficient security rights.

This one probably is probably the same as the one above. A security permissions issue. BUT it might be profile related. again because of how you migrated them from NT to Active Directory

2 sec later userenv 1502
Windows cannot load the locally stored profile. Possible causes of this error include insufficient security rights or a corrupt local profile. If this problem persists, contact your network administrator.

This one, seems as though it's looking for the Active Directory profile and can't find it. There is an account in Active Directory but no associated profile for it.

<1sec later userenv 1511

Windows cannot find the local profile and is logging you on with a temporary profile. Changes you make to this profile will be lost when you log off

This one I think is more of a problem. Not quite sure what you did here. Did you move the NT DNS to the Active Directory DNS and create a new zone for! it in Active Directory? if so, that's a problem. NT is not compatible with Active Directory. Your DNS server can act as a DNS for the NT domain, but you can't intergrate the NT domain with an Active Directory domain. They still have to remain seperate domains, thus seperate zones in DNS and not intergrated into the Active Directory DNS zones for the Active Directory domain.

Later on, we created a new DNS server on the AD domain on a DC - decommissioned the DNS server on the NT domain, and moved the IP address - x.x.x.30 - of the old NT dns server, to the DC. We bound it's DNS to x.x.x.30, and disabled it's old nic so x.x.x.30 became it's only address.

and this one is related to the one above:

0033 userenv 1054
Windows cannot obtain the domain controller name for your computer network. (A socket operation was attempted to an unreachable host. ). Group Policy processing aborted.

note: NT domain users must use the NT domain PDC or BDC to authenticate to the NT domain. They can not use the Active Directory domain to authenticate with. NT Domain users can not log on to the Active Directory domain using their NT domain profiles ..and.. their workstations have to be members of the NT domain. If their workstations are joined to the Active Directory domain, they have to use the Active Directory domain user profiles. If you want to keep NT domain user profiles in the Active Directory domain, the ADMT might not work as planned, and you'll have to use the fix corrupt profile method of creating a clean profile, then moving over the old profile settings into the new clean profile and leaving out the these 3 files from the old profile.

Ntuser.dat
Ntuser.dat.log
Ntuser.ini

by keith.abbott

"This one I think is more of a problem. Not quite sure what you did here. Did you move the NT DNS to the Active Directory DNS and create a new zone for! it in Active Directory? if so, that's a problem. NT is not compatible with Active Directory. Your DNS server can act as a DNS for the NT domain, but you can't intergrate the NT domain with an Active Directory domain. They still have to remain seperate domains, thus seperate zones in DNS and not intergrated into the Active Directory DNS zones for the Active Directory domain.
"

No what we actually did was just create an AD DNS server and move the IP address only, of the old DNS server that was serving the NT domain, over to the new DNS server on the AD domain. In this way, all of those static clients pointing to x.x.x.30 for DNS would still find a DNS server there - only now it is a different server running on an AD domain. So it's just like you said, an AD server serving up DNS to an NT domain.

"note: NT domain users must use the NT domain PDC or BDC to authenticate to the NT domain. They can not use the Active Directory domain to authenticate with. "

Right, and that is exactly how it is. The NT users authenticate to NT4 PDC and BDC. The AD users authenticate to an AD PDCE and DC.

And, I found the issue with the strange DNS Suffix! Taking a very close look at the packet captures from pinging NTPDC.NTDOM.COM vs just pinging NTPDC, I finally noticed that there was a semicolon at the end of the created fqdn in the latter case (ntpdc.ntdom.com;). I checked the search suffix on the client and even replaced that entry, though it looked fine. Still had the issue. I took a close look at the DHCP server. That's when I found the blurb about microsoft not supporting DNS suffix order - Is that true? But in any case, I noticed that the suffix search order in DHCP option 119 was separated with semicolons. That made me suspicious (not sure if semicolons are allowed there or if it has to be commas)

but then I remembered that somebody long ago(not that long ago) had specified a suffix search order in group policy. So I took a look at that and it was separated by semicolons as well, and on that one, the descrpition specifically says to separate with commas.

So I changed that, forced a gpupdate on a few clients, did an IPconfig /flushdns, repeated my pings, and this time it correctly resolved with ntpdc.ntdom.com.

One little mystery solved but I don't think it will help us with the profile issue.

btw. I have been assuming all along, that running an RSOP would show any failed permission issues for group policy. Is this true or am I swimming under a false assumption?

Thanks for all your help
k

by cg it

you want logging mode to see how the GPOs are applied to users/computers/groups of em. eg if you've got 5 GPOs applying to an OU or a combination of nested OUs you can see the result.

Planning mode allows you to see a result without applying the GPO. Great for what ifs.

RSPO doesn't really tell you conflicts per se eg this policy conflicts with this policy, because conflicts are handled by order of precedence eg local, site, domain and OU in that order so a conflict at the OU level with local machine level, OU applies with the exceptions of block and no override.

If you have 3 GPOs at the OU level and they apply in order 1,2,3 RSPO will show you the results of those GPOs being applied in that order.

And back to NT domains. Much of what admins could do in NT domains, they couldn't do in Active Directory domains. Some of the older NT guys really disliked Active Directory domains because many of the scripts, configurations, tweeks, and waaahaaas to keep NT working right no longer work. Just like the open source debate with programmers hating standardized applications. They can't do their magic or tweeks, or little code changes they like to do.

so while NT scripts are ok for NT machines, they may not be ok for AD domains. same with importing group policies from NT to AD. Especially going from NT to W2003.

and I would still try that corrupt profile method for moving NT profiles to AD profiles if desktop setting for the user environment is a necessity.

by cg it

typically, microsoft's best practice of moving from an NT domain to an Active Directory domain is to upgrade the NT domain to Active Directory. The upgrade process juggles all the seperate security domains in the NT domain, into, basically, one big security domain. While the upgrade process tries to incorporate some of the security domains into child domains, sometimes that doesn't work well. So those security domains end up like these seperate domains in a different forest. Weird but it happens, especially if there are a bunch of security domains with different names [non contigious names]. The upgrade process also changes the schema which is really at the core of the differences in NT and Active Directory. The way Active Directory names objects is different and the way those objects are cataloged is different than NT.

by keith.abbott

"you can't specify a list to use other than those you list in the options"

OK, Microsoft seemed to be saying that you could not provide a list of DNS suffix search order to clients via DHCP (option 109). But you say they really just mean that you cant OVERRIDE the Local list?

That would be fine. It just looked like you couldn't provide the suffix list at all via DHCP from their statement. In that case, what would be the point of even including the 109 option!

Also as I was going through the UserENV logs for our test user, I noticed several

"No GPO changes but couldn't read extension (xyz)'s status or policy time."

They didn't appear to be in any place significant, but I'm not sure if their presence is significant or not.

k

by cg it

using option 119.

I've not seen any large company doing this and changing the object names via DHCP option by appending suffixes I think your just asking for trouble.

What option 119 [and honestly I had to re-read the RFC again] does is change the computer name

example: NetBIOS name. Active Directory Domain name is changed.

For Group Policy [and I had to look this up] from Technet:

" By default, the primary DNS suffix portion of a computer's FQDN is the same as the name of the Active Directory domain to which the computer is joined. To allow different primary DNS suffixes, a domain administrator can create a restricted list of allowed suffixes by modifying the msDS-AllowedDNSSuffixes attribute in the domain object container. This attribute is managed by the domain administrator using Active Directory Service Interfaces (ADSI) or the Lightweight Directory Access Protocol (LDAP)".

I keep adding comments:

I just called up a collegue and asked him if he ever did this [works at HP] and he said they used to do this when they wanted to quickly move a large batch of computers to a different domain during weekend moves. What he said was, "never worked right well as you know".

by keith.abbott

OK so how do you handle it when you have applications that only refer to the computer name but are on domains different from the calling computer (eg application on computer 'itsme.mydomain.com' refers to datasource at thatserver.hisdomain.com' only as 'thatserver', other than suffix search order?

We obviously have this situation in the extreme since we currently have 2 domains sharing the same address space (until we complete the conversion - which we can't complete until we resolve the profile issue). Even then, we will still have the issue to a lesser degree.

thx,
k

by cg it

reference your info: "applications that only refer to the computer name but are on domains different from the calling computer (eg application on computer 'itsme.mydomain.com' refers to datasource at thatserver.hisdomain.com' only as 'thatserver', other than suffix search order?

Without knowing the application and how it was created [custom/standardized/open source]
and why it can not work with a new data source, tough to suggest a better way.

Moving the data source to a different domain shouldn't really be a problem with an application as long as the application is configured to use the new data source. and by that, I mean the application knows that the data source is at a particular path/location. Again, how the application does this if it's coded in or you have to specify the path during setup or changing the path...eh only the guys who made it can tell you that...

Keeping permissions, rights, might be a big problem when you move the data source as rights and permissions configured on the data source would be for an different domain.

Tough to say...

by keith.abbott

They reside in someone else's ballpark and they won't modify them.

And there is still the issue of the 2 domains. does \\myNTserver\thisshare refer to \\myNTserver.NTdom.com\thisshare or \\myNTserver.ADdom.com\thisshare.

So I am not sure we can get away from this scenario in the short term. But if it is more reliable to handle it on the host, than it is via DHCP or group policy, or more reliable to handle it some other way, we want to do that.

k

by cg it

No Control over the guilty apps
They reside in someone else's ballpark and they won't modify them.

And there is still the issue of the 2 domains. does \\myNTserver\thisshare refer to \\myNTserver.NTdom.com\thisshare or \\myNTserver.ADdom.com\thisshare.

So I am not sure we can get away from this scenario in the short term. But if it is more reliable to handle it on the host, than it is via DHCP or group policy, or more reliable to handle it some other way, we want to do that.

k

it shouldn't matter provided that your using W2K and above. you can actually turn off NetBIOS over TCP/IP and run a pure TCP/IP network with DNS......and you might want to consider WINS for pre W2K stuff and go with a simple computer name to ip address resolution rather than some Group Policy stuff that may or may not work reliably.

but if your mixing and matching domains in a non contigious namespace using trusts, and the domains are NT and W2003/2008 Active Directory, I wouldn't mix and match users unless it's done simply to access shared resources where share and NTFS permissions apply.

and after I thought about it, I wouldn't mix and match users in the domains. Either 1 or the other and they access stuff through the trust or remote access, but not access by span domains and with mapped drives, I'd screw the names and stick strictly with ip address. \\ip address\share name.. least with that, you either get a credential box popup or the trust allows access.

if it's remote, you know, least painful way, use Remote Access to control it. and that's until someone comes up with a plan to move all data and all users/compters from the NT domain to the Active Directory domain.

The 2 domains are just making a big headache and costing money.

by keith.abbott

The one suggested by alashhar - deleting all of the profile entries in the registry - and then creating a new profile. We'll have to see if it holds....

by keith.abbott

For anyone battling this type of problem, I have found a somewhat better solution than Alashars on another forum. In Alashar's solution, you - of course - wind up blowing the profile away and it requires you to rebuild the users profile - copy all their old stuff over, etc.

The following procedure avoids that and seems to fix the problem, and can be executed remotely with a little effort, without bothering the user:

1 Get the SID of logged in user that has experienced the problem - I do this by remotely executing whoami.exe using OWexec.exe (a tool similar to psexec but runs under logged in users credentials)

2 remotely connect to the user's registry and go to HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList\.

3) Note the value for ProfileImagePath

4) Check the value of ProfileImagePath for each of the other SIDs in CurrentVersion\ProfileList to see if they are the same value as noted in step 3.

a) NOTE: %systemDrive%\documents and settings\(profile) is the same path as C:\documents and settings\(profile)
b) If a duplicate value exists:
i) export the registry key and save it to c:\ drive of that machine just in case.
ii) Then Delete the key containing the duplicate path and disconnect from the remote registry.

So far this has worked for the machines I've implemented it on (over 30). But I would LOVE to have an explanation of what is going on here.

Remember that these are :
1) clients migrated from an NT4 domain to an AD domain
2) running xp and win2k
3) not running terminal services
4) a 2-way trust exists between NT and AD domains
5) Does not occur to every PC every day ? only about half of the migrated PCs have exhibited the problem. Some of those occaisionally ? some regularly.

This discussion is closed to new replies.