General discussion



By tkensc1 ·
I have 3 Domain controllers running Win2000 that intermittently get bogged down with 100% CPU utilization. Process that takes 60-80% is LSASS.EXE. I have run Trend's sysclean, scanned the servers for viruses and they are clean. All Win updates, including SP$ are installed and antivirus aptterns auntomatically update and are checked daily. 7 other servers on the network are not affected. Any ideas out there?

This conversation is currently closed to new comments.

Thread display: Collapse - | Expand +

All Comments

Collapse -

LSASS Hogging Resources

by BFilmFan In reply to LSASS.EXE Hogs Win2K CPU

I have lots of questions to ask on this issue as I have seen it many times before and assisted clients in resolving it. I can tell you that it most likely is a Windows design issue rather than a virus or malware. To help you resolve it, I need some information:

First how many clients in your environment?

What OS are the client systems running?

Any server OS (UNIX, Novell, Linux, whatever) other than 2000 in the environment?

What speed and how many processors are in the dc's? How much RAM?

Are they all global catalogs?

In general how is your AD replication model designed? Are these DC's all within the same site? Different sites? Is the KCC running? Are you part of a larger forest?

Is this Windows Small Business Server edition?

Is Exchange 2000 in this environment?

LSASS is used by AD for:

? Local Security Authority
? Net Logon service
? Security Accounts Manager service
? LSA Server service
? Secure Sockets Layer (SSL)
? Kerberos v5 authentication protocol
? NTLM authentication protocol

If the DC's are Windows 2000, then Lsass memory usage on domain controllers has two major components: one fixed and one variable.

The fixed component is made up of the code, the stacks, the heaps, and various fixed size data structures (for example, the schema cache). The amount of memory that Lsass uses may vary, depending on the load on the computer. As the number of running threads increases, so does the number of memory stacks. Lsass.exe usually uses 100 MB to 300 MB of memory. Lsass.exe uses the same amount of memory no matter how much RAM is installed in the computer. However, when a larger amount of RAM is installed, Lsass can use more RAM and less virtual memory.

The variable component is the database buffer cache. The size of the cache can range from less than 1 MB to the size of the entire database. Because a larger cache improves performance, the database engine for AD (ESENT) attempts to keep the cache as large as possible. While the size of the cache varies with memory pressure in the computer, the maximum size of the cache is limited by both the amount of physical RAM installed in the computer and by the amount of available virtual address space (VA). AD uses only a portion of total VA space for the cache. The maximum amount of VA space that AD can use is determined by the following formula:
((totalVA - 1GB) / 2)

The amount of memory that the Lsass.exe process uses increases in accordance with Active Directory usage. When data is queried, it is cached in memory. The maximum physical memory usage by the Lsass.exe process and Active Directory is 2 GB.

This means that on an x86 machine without the /3GB switch, the cache size is limited either to 512 MB or to the amount of physical RAM, whichever is smaller. With the /3GB switch, the cache size is limited to either 1 GB or to the amount of physical RAM, whichever is smaller. Note that this means that the /3GB switch begins to help as soon as the amount of physical RAM is greater than approximately 600MB (500 MB for the cache, plus approximately 100 MB for the fixed component).

Basically LSASS gets into trouble when the system cannot process NTLM authentications for security on user accounts (file system authentication, printing, etc.) and LDAP queries (searching the directory, getting a list of names for Exchange email, etc.) due to an overload of requests (DC piling on scenario) or lack of available resources to meet demand.

If you can answer the questions here, so that everyone can follow along and offer additional insight that would be great. Look forward to assisting you with this issue.

Collapse -

Network Design Overview

by tkensc1 In reply to LSASS Hogging Resources

Here is a brief outline of the network affected by LSASS.exe hogging the CPU of the 3 Domain Controllers.

There are approximately 650-700 clients in our environment.

Nine of 10 servers are running Windows 2000 Server. The tenth runs Windows Server 2003, but it is strictly an application server only- not a DC.

The clients are primarily running WinXPSP2, with some Win 2000SP4 and Win 98SE. There are also about 50 Macs on this network as well, running variations of the MAC OS. (9x, 10x)

As far as the DCs are concerned here is the data on processors and speed:

PDC: X86 Family6 Model 7 Stepping3 with 1.5GB of RAM. This is the DHCP server, Mail server ( Exchange 2000), Antivirus Information server, and Client Antivirus server.

PDC#2: x86 Family6 Model 7 Stepping8 with 512Mb RAM. It is the global catalog for the system, and also functions as an application server.

PDC#3: x86 Family6 Model 8 Stepping6 with 512MB of RAM. It is the PDC emulator and system printer server.

All PDCs are in the same room with one domain for the network.

Collapse -

Quick Reply

by BFilmFan In reply to Network Design Overview

Are all of those single processors?

The devices which are not natively AD aware (read that to mean the MAC's, WIN NT 4 servers and workstations, Win 98 SE) boxes will be doing some heavy NTLM authentication.

Do you also have nested security groups? If you do, then those systems will make what is called a "deep LDAP query" when sorting members of groups as the MemberOf is not a true attribute, but a pointer.

My analysis would be:

Insufficient amount of RAM available for deep query LDAP queries. I would advise having no less than 4 GIG of Ram in EVERY DC.

If you can afford to go multi-processor this would also lighten the load.

You should note lsass using about 80% of memory resources isn't all that unusual, but you could have an issue in the event a major directory event (install a new GC, add a domain, demote a DC, swap a FSMO role, etc) had to take place with that sort of traffic level already in place.

There have been numerous patches made to lsass.exe depending on what version of Windows you are running. I highly recommend you contact Microsoft's PSS support and open a case with them. When you tell the tech router that you are
calling for an lsass patch they should not charge you for the case.

In addition, there is a tool available called ADPerf, which takes some experience to learn to read, but is invaluable in troubleshooting these issues.

Collapse -

Updated Information

by tkensc1 In reply to Quick Reply

This problem began on Jan. 31st, 2005. No significant hardware or software changes were made to the PDCs before or since, other than normal antivirus updates and Windows updates. The DCs ( all single processor equipped) functioned at very low CPU utilization until that date, when our network was hit by the WORM_AGOTBOT.AHY. All servers and workstations were "syscleaned" and scanned and checked for current virus protection and cleanup templates. This brought all servers except the DCs back to normal operation. We had the same amount of RAM before the virus infection as we do now- and the system performed very well prior to that time. What would the virus/worm infection have done to make the amount of RAM insufficient?

A case has already been opened with Microsoft and the lsass.exe patch has already been applied- no change.

Collapse -

Viral Infection

by BFilmFan In reply to Updated Information

Sounds as if you still have a viral infection running somewhere then. Do you have the capability to sweep network traffic looking for suspicious port activity?

Collapse -

No Virus

by tkensc1 In reply to Viral Infection

I had all 3 DC's scanned with process explorer and other utilities provided by our antivirus provider. All 3 machines came back clean. All 3 are still running at near 100% CPU sustained, and last weekend, the activity flooded over into our hardware firewll and pegged that at 95%. If this is a virus- the antivirus providers haven't gotten a fix for it yet, despite updates 2-3 times per day.

Collapse -

Problem Solved!

by tkensc1 In reply to No Virus

Thanks to analysis of userdump files by Microsoft, the problem in our 3 Win2k domain controllers was traced to a memory leak in the LSASS.exe process- a known issue in Win2k. Microsoft sent us a beta hotfix that solved the problem, and our 3 DC's are now doing just fine.

What cause the memory leak? Several of the latest worms & trojans which gain entrance to a firewalled system through the IRC chat port 194. We have tempoarily blocked that port with our antivirus product, and are manually closing off that port in the services file located in C:\WiNNT\System32\Drivers\etc folder. Open the file with Notepad, and just comment out the IRC line and save the file.

Collapse -

Can you provide fix information?

by In reply to LSASS.EXE Hogs Win2K CPU

Hi tk,

can you update and advise the specifics around this fix that was provided to you by Microsoft. Do you have a KB number or link, or perhaps you can specify the actual issue and cause. Is this virus related? At the very least, do you have a name of someone at MS who could provide more information? Feel free to send me a personal message if you prefer.


Collapse -

Did you find out this info?

by tperault In reply to Can you provide fix infor ...

Hey Curious,

Did you happen to get the KB # or the link for this info? If so could you also let me know?


Related Discussions

Related Forums