Saturday, July 30, 2016

Windows 10 Registry Size, Number of Keys, Values




Registry Size & Number of Keys (Key/Value Pairs)


With a new operating system Windows 10 comes a brand new registry and I got asked by a client in a discussion;  "Well how big is it?; and; Why does it take so long to search?" I thought this was trivial to answer, but not easily found, as it turns out. So here is the definitive answer.
 

A little backgrounder on what the registry is;
The Windows Registry is a hierarchical database that stores low-level settings for the Microsoft Windows operating system and for applications that opt to use the Registry. The kernel '(OS)'device driversservicesSecurity Accounts Manager (SAM), and user interface can all use the Registry. The Registry also allows access to counters for profiling system performance, aka 'performance data' (Wikipedia). 

Note, most apps/programs add registry entries, but when they are uninstalled some companies are lazy and do not clean up there tracks. Hence registry size tend to grow over time. Remnants remain in the registry and hence tools like registry cleaner and optimizers were born. Here's a complete set of free open source registry utilities use them, but they are snake oil according to Malwarebytes Labs.

Detailed look at registry at http://www.techsupportalert.com/content/deeper-windows-registry.htm.



Results to these 3 common questions about the Registry


Q1) How big is it ? 


Method 1 : Export Registry to a .reg file

For our needs, we are going to work with an exported registry file. This allows use to avoid pesky permissions and speed constraints when using the actual registry database.

Layout of a exported registry file using Registry Editor (
Regedit.exe)


; Sample exported registry file
[HKEY_LOCAL_MACHINE\SOFTWARE\Office]     ; path to subkeys, like directories 
    @="Value"      ; @ is same as (default) key name in registry, key/value pair                                      "Key"="Value"      ; a key/value pair


Methodology Overview for Registry Stats

  1. Export entire registry to file.
  2. Count key/value pairs in the file with lines that start with @ or "
  3. Count subkey paths in the file with lines that start with [

With a fresh install of Windows 10 Pro the registry database contains approx 3 million subkeys and key\value pairs combined. 
Entire registry exported .reg size in 716.82 Mb on disk.


New Windows 10 Pro install with Office 2016 Registry Counts


Total Number of Lines 5,789,565 approx 6 Million
Total Number of Key/Value Pairs 1,973,233 approx 2 Million
Total Number of Subkey "Paths" 1,069,402 approx 1 Million
Total Number of Empty lines 2,746,930 approx 3 Million
All Keys+Values 3,042,635 approx 3 Million

























Method 2 : Dureg.exe data size

Dureg.exe is a command-line utility that you can use to determine how much data is stored in the registry, or in any registry subtree, key, or subkey. You can also use this tool to search the registry or any registry subtree for occurrences of a specific text string. 


Download Dureg.exe installer (x86 app and is reading Wow6432Node)


C:\WINDOWS\system32>dureg /a
Size of HKEY_CLASSES_ROOT   :   45,914,070
Size of HKEY_USERS          :   49,949,370
Size of HKEY_LOCAL_MACHINE  :   86,644,163

    Total Registry data size:    182,507,603

But we are missing 2 Hives (they are just shortcuts, so we don't count them)


  1. HKEY_CURRENT_CONFIG is a shortcut to the HKEY_LOCAL_MACHINE hive. More specifically, to that hive's \SYSTEM\CurrentControlSet\Hardware Profiles\Current\ registry key.
  2. HKEY_CLASSES_ROOT is actually a copy (or an alias, as these copied keys are called) of the following HKEY_LOCAL_MACHINE hive with path HKEY_LOCAL_MACHINE\Software\Classes
DuReg.exe counts data, not space. Because the space any data occupies varies with the storage method and amount of free space available, the size of data and the size of space consumed are not equal. Also, because it measures data, DuReg does not account for fragmented free space in the registry.

Dureg.exe estimates 182.5 Mega data size. 

Method 3 : Ru.exe estimate size on disk


Using Sysinternals Registry Usage Tool  



RU Command Size (bytes)
ru HKEY_LOCAL_MACHINE 196,273,956
ru HKEY_CURRENT_USER 15,214,237
ru HKEY_USERS 41,702,481
Totals 253,190,674


or 253.1 Mb estimated size on disk.



Using Sysinternals Registry Usage Tool  gives counts as well


RU Command               Keys           Values
ru HKEY_LOCAL_MACHINE 697,274 1,321,806
ru HKEY_CURRENT_USER 35,815 112,205
ru HKEY_USERS 100,396 303,160
Totals 833,485 1,737,171
All Keys+Values 2,570,656

Method 4 : Actual size on disk


Compare this to actual Registry Hives on disk using Windows Explorer


Representative Hive Files on disk size (bytes) size (bytes)
HKEY_CURRENT_USER Ntuser.dat, Ntuser.dat.log 14,942,208
HKEY_LOCAL_MACHINE\SAM Sam, Sam.log, Sam.sav 65,536
HKEY_LOCAL_MACHINE\Security Security, Security.log, Security.sav 28,672
HKEY_LOCAL_MACHINE\Software Software, Software.log, Software.sav 218,890,240
HKEY_LOCAL_MACHINE\Components Components, Components.log, Components.sav 48,234,496
HKEY_LOCAL_MACHINE\System System, System.alt, System.log, System.sav 15,728,640
HKEY_LOCAL_MACHINE\BCD00000000 BCD-Template, BCD-Template.log 28,672
HKEY_LOCAL_MACHINE\ELAM ELAM, ELAM.log1 32,768
HKEY_LOCAL_MACHINE\Drivers Drivers, Drivers.log1 5,357,568
HKEY_LOCAL_MACHINE\BBI BBI, BBI.log1 524,288
HKEY_LOCAL_MACHINE\VSMIDK VSMIDK, VSMIDK.log1 4,096
HKEY_LOCAL_MACHINE\userdiff userdiff, userdiff.log1,userdiff.log2 8,192
HKEY_LOCAL_MACHINE 288,903,168 288,903,168
HKEY_USERS Default, Default.log, Default.sav 14,942,208
Total  318,787,584
(sums do not include .log, log1, .sav)

Total registry size of 318.8 Mb on disk




Method 5 : Database size

Compare this to the actual Registry database size of 352 Mb


Registry Hierarchical Database File Size approx 0.3 Gb



Get Registry Age and Size running on  Microsoft Windows 10 Pro

CurrentSize (Mb) : 352
MaximumSize (Mb) : 4095
FreeSize (Mb)    : 3743
PercentFree      : 91.4041514041514
Created          : 9/28/2016 11:40:47 PM
Age              : 215.21:15:06.4100071



Registry Age and Size Powershell script source




Q2) Why does it take so long to search the Registry ? 

Short Answer: Because the registry is a hierarchical database which is "like" a tree structure and implementation is not well know. Allot of controversy around this with some basic saying "it's essentially a flat database or list."  However, it's more complicated than that, digging deeper lead the famous Mark Russinovich who is creator of Sysinternals the advanced system utilities and technical information site.
Inside the Registry by Mark Russinovich states on disk, the Registry isn't simply one large file but a set of discrete files called hives. Each hive contains a Registry tree, which has a key that serves as the root (i.e., starting point) of the tree. The Configuration Manager logically divides a hive into allocation units called blocks in much the same way that a file system divides a disk into clusters. By definition, the Registry block size is 4096 bytes (4KB). Blocks hold cells. The Registry data that a hive stores in containers called cells. A cell can hold a key, a value, a security descriptor, a list of subkeys, or a list of key values. A field at the beginning of a cell's data describes the data's type.he distinction between cells, bins, and blocks can be confusing, so let me give you an example of a simple Registry hive layout. The sample Registry hive file in Figure 1 contains a base block and two bins. The first bin is empty, and the second bin contains several cells. Logically, the hive has only two keys: the root key Root, and a subkey of Root, Sub Key. Root has two values, Val 1 and Val 2. A subkey-list cell locates the root key's subkey, and a value-list cell locates the root key's values. The free spaces in the second bin are empty cells. The figure doesn't show the security cells for the two keys, which would be present in a hive. This is the representation on disk.

look like a link list to me on disk

There have been calls to move this into a relational  database. For windows programs using language C# lookups, using a dictionary structure is the time required is flat, an O(1) constant time complexity. The List has an O(N) linear time complexity, in worst case scenario. That means you have to look through all 2M key/value pairs plus keys (1M) for a total of a linear search through 3M items. Registry feels like a link list when I search. It can find a key quickly or take forever, it feels pretty linear.
  • Note: It's a little way more complicated than that, since the hive is kept in memory and with its own structure.

    From Inside the Registry by Mark Russinovich states; 

    To deal with non-contiguous memory buffers storing hive data in memory, the Configuration Manager adopts a strategy similar to what NT's Memory Manager uses to map virtual memory addresses to physical memory addresses. The Configuration Manager employs a two-level scheme, which Figure 2 illustrates, that takes as input a cell index (i.e., a hive file offset) and returns as output both the address in memory of the block the cell index resides in and the address in memory of the bin the cell resides in. Remember that a bin can contain one or more blocks and that hives grow in bins, so NT always represents a bin with a contiguous memory buffer. Therefore, all blocks within a bin occur within the same portion of a paged pool.
    Cc750583.inreg02(en-us,TechNet.10).gif

Q3) What is Windows 10 Pro Registry Path  Frequency Distribution?

A sample subkey depth calculation for the following registry key path 

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Run\AutorunsDisabled

has a depth of 7. 



How many Windows Registry Types are there and what are their counts? 
  • Here's a Windows Registry Types frequency counts for a typical old Win7 machine with Office installed.



































These are all the windows registry types that appear in .reg file see RegtoText for more info.


Method 1 : Detailed Methodology for Calculation

  • 1. Export Your entire registry to a file. This backup up your registry ! Do it now. 


















  • 2. Here's a example of more complex Registry export file.
        Take note of the hex and dword values span multiple lines. 
Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VolumeCaches\Windows Error Reporting System Queue Files]
@="{C0E13E61-0CC6-11d1-BBB6-0060978B2AE6}"
"FileList"="*.*"
"Flags"=dword:0000007d
"Folder"=hex(2):25,00,41,00,4c,00,4c,00,55,00,53,00,45,00,52,00,53,00,50,00,52,\
  00,4f,00,46,00,49,00,4c,00,45,00,25,00,5c,00,4d,00,69,00,63,00,72,00,6f,00,\
  73,00,6f,00,66,00,74,00,5c,00,57,00,69,00,6e,00,64,00,6f,00,77,00,73,00,5c,\
  00,57,00,45,00,52,00,5c,00,52,00,65,00,70,00,6f,00,72,00,74,00,51,00,75,00,\
  65,00,75,00,65,00,00,00
"IconPath"=hex(2):25,00,53,00,79,00,73,00,74,00,65,00,6d,00,52,00,6f,00,6f,00,\
  74,00,25,00,5c,00,73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,00,77,\
  00,65,00,72,00,75,00,69,00,2e,00,64,00,6c,00,6c,00,2c,00,31,00,00,00
"Display"=hex(2):40,00,25,00,53,00,79,00,73,00,74,00,65,00,6d,00,52,00,6f,00,\
  6f,00,74,00,25,00,5c,00,73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,\
  00,77,00,65,00,72,00,2e,00,64,00,6c,00,6c,00,2c,00,2d,00,32,00,39,00,39,00,\
  00,00
"Description"=hex(2):40,00,25,00,53,00,79,00,73,00,74,00,65,00,6d,00,52,00,6f,\
  00,6f,00,74,00,25,00,5c,00,73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,\
  5c,00,77,00,65,00,72,00,2e,00,64,00,6c,00,6c,00,2c,00,2d,00,33,00,30,00,30,\
  00,00,00
"StateFlags"=dword:00000000
  • 3. Count key/value pairs in the file with lines that start with @ or "
  • 4. Count subkey paths in the file with lines that start with [
  • 5. Done. 


Explore the Registry : Related Tools


  1. My RegtoText is a command line utility that converts convoluted hex values in Windows Registry file (.reg) into a human readable text (.txt) file.  Specifically it identifies the 14 hex formats and converts them to ASCII or UTF-8.
  2. My RegViewer a read-only Windows registry viewer, safe and easy to use 

2 comments:

  1. Your post is excellent. Anything to update for the end of 2017?

    ReplyDelete
    Replies
    1. https://goo.gl/k5KD19 - you can try my new simple mismatch quotes inspector:) Cheers

      Delete