Windows 10's memory management feature is far superior to its predecessors, but it's also misunderstood. Here's where to see the memory compression feature at work, and how to monitor it.
I recently received an email from a reader who saw my How to use Task Manager's Processes tab to troubleshoot issues in Windows 10 article and was wondering if I could help her to troubleshoot a memory problem on her Windows 10 laptop.
After I analyzed what she was describing, I was able to assure her that what she was seeing was a feature called Memory Compression and not a problem to worry about. There was nothing wrong with her laptop. Since other Windows 10 users have the same concerns as she did, I decided to write an article describing Windows 10's Memory Compression feature.
The perceived problem
My correspondent told me that when she first started using Windows 10, she was regularly witnessing the System process showing a high percentage in the heat map and that it was consuming a great deal of memory, similar to the Task Manager Processes tab shown in Figure A.
The System process is consuming a good chunk of memory.
She then told me that several Windows updates later she would see a process called System and compressed memory jump to the top with a high percentage in the heat map and that it too was consuming a great deal of memory, similar to the Task Manager Processes tab shown in Figure B.
The System and compressed memory process is consuming a good chunk of memory.
She was worried that these processes were consuming all her memory and that her system performance was suffering. However, in both cases, she was witnessing the Memory Compression feature at work, which is actually a solution to a long-standing problem, and nothing to be concerned about.
Memory management overview
Memory compression is a Windows memory management technique introduced in Windows 10 and designed to improve the responsiveness of the operating system during heavy use. And, as you can imagine, it's the most recent of a long line of memory management features. To get a grip on exactly what memory compression is all about, it will be worthwhile to step back in time and take a look at the evolution of Windows memory management technology. (Keep in mind that in this overview, I won't get into the heavy duty technical aspects of memory management, nor will I attempt to pinpoint the version of Windows in which each new memory management feature was introduced.)
SEE: Quick glossary: Storage (Tech Pro Research)
Page file system
For as long as Windows has been around, memory management has relied on the page file system. Here's how it works: When the amount of memory in use by all the running processes exceeds the amount of RAM available in the system, the operating system will begin moving pages of memory out of RAM and temporarily storing them on the hard disk, thus freeing that RAM for other uses.
When a running process needs to access the memory that has been temporarily moved to page file, the operating system locates the page containing the needed data and returns it to RAM. As it does so, the operating system will move other pages of memory out of RAM to the page file in order to make room for the data that it just returned to RAM.
This moving of memory pages is referred to as swapping and the paging file is often referred to as the swap file. Because the pages of memory are kept intact as they are stored on the hard disk, the term virtual memory is often used.
Regardless of what it is called, this swapping requires a lot of overhead.
To help reduce the overhead involved with swapping pages of memory back and forth between RAM and the page file, Microsoft came up with a system called Prefetch. Under this system, the operating system loads key pieces of data and code from disk into memory before it's actually needed.
More specifically, a Cache Manager monitors the data being moved between the disk and RAM and between RAM and virtual memory when various applications are loaded. As the Cache Manager monitors these occurrences, it constructs maps of the directories and all of the files that were referenced for each application or process.
Once these map files have been created, the Cache Manager uses them to improve efficiency when loading applications. More specifically, the Cache Manager will intercept every process or application that is about to be loaded and will check to see if there is a corresponding map.
If there is, the Cache Manager will call on the file system to immediately access the directory and files referenced in the map. The Cache Manager will then alert the Memory Manager and tell it to use the information in the map file to load data into memory.
Once this prefetch operation is complete, the Cache Manager will allow the application to continue loading. As the application does so, it will find the majority of the files and data that it needs are already available in memory. This, of course, reduces the amount of disk access and allows the application or process to load or respond much faster.
As I've explained, Prefetch improves efficiency by loading the majority of the files and data needed by an application or process into memory so that that can be accessed very quickly when needed. SuperFetch takes this system one step further.
In addition to constructing the map files I described earlier, SuperFetch also constructs profiles of the applications that you use. These profiles include information about when and how often you use your applications. SuperFetch keeps track of the applications in your profile and notes when any prefetched data is moved out to the page file. Superfetch will then monitor the progress of the application that caused the prefetched data to be moved out to the page file. As soon as that application is done, SuperFetch will pull the prefetched data back into memory. Thus, when you go to access the application, the prefetched data will again be available in memory and the application will be able to very quickly pick up right where it left off.
SEE: Windows 10: The smart person's guide (TechRepublic)
As I've mentioned, the Windows operating system will always make use of virtual memory, no matter how much physical RAM is installed in the system. I've also told you that one of the enhancements in SuperFetch is that it retrieves prefetched data from the virtual memory page file to physical memory as soon as possible to ensure the continuity of the performance enhancement. However, because the page file exists on the hard disk, which is less responsive than physical memory, SuperFetch is still going to be hindered by the time that it takes the hard disk to respond to its retrieval operation.
To improve SuperFetch's retrieval operation, Microsoft took advantage of the fact that large capacity USB flash drives were widely available, relatively inexpensive, and very efficient--not as efficient as physical memory, but much faster than a hard disk.
ReadyBoost creates cache file on the USB flash drive that SuperFetch uses to store copies of its map files and application profiles - it also stores the same data on the hard disk. That way, if you remove the USB flash drive from your computer, SuperFetch can still function - it just gets the data from the hard disk.
While SuperFetch made dramatic improvements to the system, there's a catch: it only uses the ReadyBoost cache to retrieve non-sequentially stored data. If the data is stored sequentially, SuperFetch retrieves the data from the hard disk cache. The reason for this is that hard disks are faster than flash drives during sequential reads, but flash drives are faster during non-sequential reads. More specifically, when it comes to reading non-sequential data, the physical movement of the hard disk read/write heads adds latency to the read operation.
Now that you have an inkling of how Windows memory management features have evolved over time, you'll be able to get a better feel for how memory compression works and why it provides better results than its predecessors. Here's how it works:
As you use your computer, the memory manager monitors the applications that you load. When the amount of memory in use by all the running processes reaches a certain percentage of the amount of RAM available in the system, the operating system will begin compressing pages of memory so that they take up space. Once compressed, the memory manger moves pages into another part of memory called the compression store. When a running process needs to access the memory that has exists in the compression store, the operating system locates the page and decompresses it. Since the data is never written to the hard disk and in fact never leaves memory, the whole swapping process is very fast and efficient.
Not only is the process fast, but the space-saving aspect of the compression is considerable. In fact, pages of memory are compressed to about 30% of their original size. This means that the process will essentially return 70% of the previously used space to be used by other applications.
Of course, the compressing and decompressing of memory requires a lot of CPU effort. However, since modern systems have multiple cores, the operating system can delegate this task to other cores that might otherwise be idle. Furthermore, the compression operation runs with a low priority, so it won't take the CPU away from more pressing tasks. And, the decompression is very fast, so it won't hog CPU cycles.
Keep in mind, that even with this new system, Windows 10 still must occasionally swap memory out to the page file on the hard disk. According to Microsoft, this should be rare and only occur on systems with a small amount of memory. Even so, this process is more efficient than in previous versions because the pages that are moved to the page file are highly compressed. Since they are compressed, moving them back and forth between the page file and memory is very fast. So in the long run, Memory Compression is much better than any of the previous memory management techniques that depend of writing and reading data from the hard disk.
Monitoring memory compression
In earlier versions of Windows 10, the compression store existed in the System process, which is why you'd see the System process using lots of memory on the Processes tab. As compressed files were being written to the compression store, you'd see the System process requiring a high percentage of memory resources. This caused a lot of folks, like my reader, to be concerned.
In a later version of Windows 10, Microsoft created a separate process called System and compressed memory and used it to show the compression activity. While more aptly named, the jump in memory use still caused a lot of folks to be unnecessarily concerned.
In a more recent version of Windows 10, Microsoft again refined the name of the process to Memory Compression, but also hid it such that it no longer shows up in Task Manager's Processes list.
To see the Memory Compression process, you can use the PowerShell and the Get-Process -Name "Memory Compression" command, as shown in Figure C. However, as you can see, this view isn't very easy to interpret.
You can view the Memory Compression process with PowerShell.
If you want to see how much of the memory in use on your system is compressed, access the Performance tab in Task Manager and select Memory. You'll then see that the In use section shows how much memory is compressed. As you can see in Figure D, on my example system, 5.2 GB of memory is in use, and of that 782 MB is compressed.
You can see how much memory is compressed in Task manager.
If you hover your mouse pointer over the Memory composition graph, you'll get more detailed information. This level of detail reveals that on my example system, the 782 MB of compressed memory would normally take up 2449 MB (2.39 GB) of space in memory. So there is a savings of 1667 MB (1.62 GB) due to compression. In other words, there is now 1.62 of additional free space in memory for other applications.
- Windows 10 performance and troubleshooting tips: Run faster, smoother, longer (ZDNet)
- How to use Task Manager's Processes tab to troubleshoot issues in Windows 10 (TechRepublic)
- How to get more information from Windows 10's Details tab by adding columns (TechRepublic)
- How to set up a mobile hotspot with Windows 10 (TechRepublic)
- 6 tips for cleaning your computer's hard drive (TechRepublic)
- Top Windows 10 questions: How to install, secure, upgrade, get it for free (ZDNet)
What's your take?
What do you think about the memory compression feature in Windows 10? Share your thoughts with fellow TechRepublic members.