Saturday, July 30, 2016

RegtoText - NEW command-line executable converts Windows Registry file to human readable text

RegtoText is a command line utility that converts a Windows Registry exported file (.reg) into a human readable text (.txt) file. Hex numbers are converted into ASCII characters when possible. Conversion can be challenging since registry key can accept any binary format, so heuristic and probabilistic methods are used to decode values when possible to ASCII.

This tool target as forensic (FBI,CIA, Antivirus Co)/management/educational tool to quickly search and eyeball the entire registry file for encoded values that are suspicious. Registry keys could hold persistant malware signatures (like Poweliks), back-doors or simply hidden secret messages. Most commonly, keys contain   foreign languages encodings that can be spotted more efficiently with human eyes. Furthermore, once this file is decoded using RegtoText, it is searchable as a human readable text file and can be indexed in any internal forensic exploit search engine/database.















RegtoText converts a Registry exported file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Windows Registry Editor Version 5.00

HKEY_CURRENT_USER\Control Panel\Appearance\New Schemes\4\Sizes\0]
"DisplayName"="@themeui.dll,-2019"
"Flat Menus"=dword:00000000
"Font #0"=hex:f5,ff,ff,ff,00,00,00,00,00,00,00,00,00,00,00,00,bc,02,00,00,00,\
  00,00,00,00,00,00,00,54,00,61,00,68,00,6f,00,6d,00,61,00,00,00,00,00,00,00,\
  00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,\
  00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00
"Size #0"=hex(b):01,00,00,00,00,00,00,00
"Size #9"=hex(b):00,00,00,00,00,00,00,00
"Color #0"=dword:00c8d0d4
"LegacyName"="@themeui.dll,-854"

into a human readable text file either UTF-8 or ASCII (for ASCII ?? and odd characters are removed)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
RegtoText Windows Registry Conversion Version 5.00 (sample UTF-8 export)

[HKEY_CURRENT_USER\Control Panel\Appearance\New Schemes\4\Sizes\0]
"DisplayName"="@themeui.dll,-2019"
"Flat Menus"=[REG_DWORD] 0
"Font #0"=[REG_BINARY] ??      '     Tahoma                         
"Size #0"=[REG_QWORD] 1
"Size #9"=[REG_QWORD] 0
"Color #0"=[REG_DWORD] 13160660
"LegacyName"="@themeui.dll,-854"


REGTOTEXT HELP


.FILENAME
RegToText.exe 

.VERSION

Version 2.2016.214.373 (Update Sept 20, 2016)

.SYNOPSIS 
Parses a valid Windows registry exported file (.reg) and translates indecipherable hex and decimal values into a human readable text file.

.PURPOSE
The aim of this command-line executable is to make a human readable registry file. This greatly aids in searching and understanding the Windows Registry, key for developers.

.DESCRIPTION 
RegToText windows console application deciphers unreadable portions of registry file to text. Firstly, it checks for a valid Windows registry file ending with file extension (.reg). Then it validates file header for ""Windows Registry Editor Version 5.00"" for Windows 2000, ME, XP,7, Vista, 8, 8.1, 10+, Server 2003+ or ""REGEDIT4"" for Windows 98, NT 4.0 and Server 2000-. Passing this, the process will begin to translates all the hexadecimal and decimal values into output Unicode text file. Output is written out in 250 line chunks. Upon premature or cancellation, output file will contain up-to the last chunk written out. Output encoding can be UTF-8 or ASCII. Output encoding can be UTF-8 or ASCII. All non-printable characters are cleansed for ASCII, read ENCODING notes for details. Encoding choice can drastically affect file output size. 

All the registry types are translated listed as follows;


"value" alias hex(1) Default or blank String value data with escape characters
hex alias hex(3) REG_BINARY  Binary data (any arbitrary data)
dword alias hex(4)
REG_DWORD / REG_DWORD_LITTLE_ENDIAN
A 32-bit unsigned integer coded in little-endian format
hex(0) REG_NONE No type (the stored value, if any)
hex(1) REG_SZ A string value, normally stored and exposed in UTF-16LE (when using the Unicode version of Win32 API functions), usually terminated by a NUL character
hex(2) REG_EXPAND_SZ An “expandable” string value that can contain environment variables, normally stored and exposed in UTF-16LE, usually terminated by a NUL character
hex(3) REG_BINARY Binary data (any arbitrary data) including variable byte encoded UTF8 values
hex(4)
REG_DWORD_LITTLE_ENDIAN equivalent to REG_DWORD 
A 32-bit unsigned integer coded in little-endian format
hex(5) REG_DWORD_BIG_ENDIAN A 32-bit unsigned integer coded in big-endian format
hex(6) REG_LINK A symbolic link (UNICODE) to another Registry key, specifying a root key and the path to the target key
hex(7) REG_MULTI_SZ A multi-string value, which is an ordered list of non-empty strings, normally stored and exposed in UTF-16LE, each one terminated by a NUL character, the list being normally terminated by a second NUL character.
hex(8) REG_RESOURCE_LIST A resource list
hex(9) REG_FULL_RESOURCE_DESCRIPTOR A resource descriptor
hex(a) REG_RESOURCE_REQUIREMENTS_LIST A resource requirements list
hex(b) REG_QWORD_LITTLE_ENDIAN equivalent to REG_QWORD 
A 64-bit integer little-endian (introduced in Windows XP)

.LIMITATIONS

  1. Does not decode Darwin Descriptors (DD), perhaps in future enterprise version. Vote for it in comments section.(DD is basically an obfuscated GUID a very basic tool is available for download here)
  2. Does not unpack packed GUIDs, perhaps in future enterprise version. Vote for it in comments section.(here's a Powershell script to available for download here)
  3. Does not ROT-13 decode the UserAssist key (HCU\Software\Microsoft\Windows\CurrentVersion \Explorer\UserAssist). Vote for it in comments section.(decode at http://rot13.com/)
  4. Does not perform language encoding translations. Vote for it in comments section.


.REQUIREMENTS
32-bit app which requires .NET Framework 4 Client Profile.

.64 BIT REGISTRY KEYS

The registry in 64-bit versions of Windows is divided into 32-bit and 64-bit keys. Many of the 32-bit keys have the same names as their 64-bit counterparts, and vice versa.
To open registry 64bit version run %systemroot%\syswow64\regedit from CMD as Admin.

Refer to https://support.microsoft.com/en-ca/kb/305097 to extract 64-bit keys to .reg file. RegtoText decodes 64-bit keys as 32-bit keys. There is no difference.

.HIDDEN REGISTRY KEYS - SECURITY & SAM
Some of the security and core system related keys are hidden from user even when part of an administrator group cannot see these special keys.

Here are some of the such hidden registry keys

HKEY_LOCAL_MACHINE\SECURITY
HKEY_LOCAL_MACHINE\SAM

SECURITY registry key stores all the system policy and LSA secrets related information.  SAM registry key has details for user accounts along with LM/NTLM password hashes for each user.

There are many ways we can view these hidden registry keys. We can use psexec.exe tool (part of pstools package from sysinternals) to launch the regedit.exe as system account as shown below.


psexec.exe -s -i regedit.exe

.ENCODING
Characters outside the ASCII or UTF8 range are stripped. NON-PRINTABLE less than decimal 31 are stripped for ASCII encoding. UTF8 preserves more of the original source content, but a cost of larger output file size. More importantly, UTF8 encoding will pass allot of unreadable characters and non-printable characters that may cause issues when scrolling large files in text editors. ASCII allows for maximum readability with intention of one line per registry key and side benifit of space savings. Large files over 1G benefit tremendously when loading ACSII text editors for scrolling and searching.

.TEXT EDITORS

Notepad and Notepad++ will not load 1 G+ files. Textpad (memory lim), Notepad Light (upto 2G) and UltraEdit (claims 2^64-1G) will load file over 1G+ files.

.PERFORMANCE 
Tested on 4.25M rows in 18 mins, 24 secs. Processing 921,572 subkeys and 2,344,590 key/value pairs.

.USAGE
RegtoText.exe [/h] [/v] [/s] inputfile.reg [/o:filename.txt] [/e:{UT8F|ASCII}]

.ARGUMENTS
[drive:][path]inputfile.reg            1st argument required
                                       Input registry file. If path omitted, default to current path. 
.FLAGS
(order not important)
/h|/help                               Help
/v|/version                            Version
/s|/silent                             Silent
/l|/license                            License
/e|/encoding:{UTF8|ASCII}              Output encoding. If omitted, default value:'UTF8'.

/o|/output:[drive:][path]filename.txt  Output text file. If omitted, default value:'inputfile.txt

.INPUT
Must be valid exported registry file from REGEDIT.exe ending in .reg

.OUTPUT
Creates a Unicode text file ending in .txt extension. If exists prompt to delete ? No, creates a timestamped file. Hexadecimal and decimal values are decoded using according /e flag.

.EXAMPLE 
regtotext c:\Users\MDC\Documents\myfullregistryBCK.reg /e:ASCII

.AUTHOR 
metadataconsult@gmail.com (Metadata Consulting, ON, CDN) July 30, 2016

.LICENSE
Read Full License Agreement use /l FLAG OR pipe into a text file using 'regtotext /l > RTTLic.txt' to read in Notepad.


Download RegtoText demo version 2 (update Sept 2016). Read demo license.


For a commercial licensed version contact metadataconsult@gmail.com



Commercial version sample run on a new Windows 10 Pro install with Office 2016. 


2 comments:

  1. Is there really a market for this tool?

    Please note that it's possible to export to .txt with RegEdit as well (although I'm not sure about the UTF-8 encoding part...).

    ReplyDelete
    Replies
    1. This tool only works from the exported .txt file from RegEdit! That is the whole point! Read the standard Regedit text file is very is difficult, unless you are "Neo" (from Matrix) an can read hexadecimal on the fly.

      This tool target as forensic/management/educational tool to quickly eyeball the entire registry file for encoded values that are suspicious. Registry keys could hold malware signatures, back-doors or simply hidden messages, but most commonly are foreign languages encodings can be spotted more efficiently with human eyes. Furthermore, once this file is decoded using my tool, it is searchable as a human readable text file and can be indexed in any internal forensic exploit search engine/database. So yeah, the applications of this tool is "huge".

      Delete