Friday, September 08, 2006

Partial file matching in host intrusion prevention systems

A few weeks ago Jesse Kornblum released the SSdeep [1] tool. The main propose of this tool is to identify similar files by calculating hashes and comparing those hash values to the known values computed previously and stored in database.
he big difference between SSdeep and other well-known tools to generate hash values (like md5sum) is that SSdeep calculates hash values for small chunks of target file. So if someone modifies only few bytes of a file, the new calculated hash value will be similar to the previous one [2].


The main features (altered document matching and partial file matching) of the SSdeep are very helpful during forensic analysis. But you can also use partial file matching feature in host intrusion prevention systems. You can use this function to disable execution of specific programs.

It was rather useless to use “normal” hash values in IPS to prevent execution because such solution can be easily cheated. After changing even one byte in executable file the new hash value is completely different. As you can guess there are a lot of places in an executable file which can be modified and the exe file still will be executed without any problems.

Before one byte modification:

C:\ssdeep>md5sum gg.exe
\0323de930ed3e8e0552843db7e16dab7 *C:\\ssdeep\\gg.exe

After one byte modification:

C:\ssdeep>md5sum gg.exe
\bfa5aed4078c2a316786c1e7cb1e4f8e *C:\\ssdeep\\gg.exe

As mentioned above the SSDeep generates has values for small blocks of target file. So few modifications of target file will not change the whole value of generated hash as it is presented below:

Before modifications:

C:\ssdeep>ssdeep -l gg.exe

C:\ssdeep>ssdeep gg.exe > sum.txt
C:\ssdeep>ssdeep -m sum.txt gg.exe
C:\ssdeep\gg.exe matches C:\ssdeep\gg.exe (100)

After few modifications in .rsrc section:

C:\ssdeep>ssdeep -l gg.exe

C:\ssdeep>ssdeep -m sum.txt -p gg.exe
C:\ssdeep\gg.exe matches C:\ssdeep\gg.exe (91)

Additionally, the percentage value of similarity is generated (value in brackets).
By setting the value to 60 or 70 we can implement quite effective method of blocking execution of specific files.

This solution could block the execution of particular program and even new versions of it because very often new releases are based on the previous one.

Useful links:



Tuesday, August 22, 2006

Grsecurity and forensic analysis

A few weeks ago a new version of grsecurity 2.1.9 was released [1]. It is worth to mention about it because one new features affect how Linux physical memory forensic analysis will be performed.

Firstly, all physical memory pages which are freed are overwritten. During freeing page frames, a new PaX feature zeroes out them. It means that it will be impossible to recover content of pages such as memory mapped files from memory images which represent /dev/mem or /proc/kcore. Still, we can use methods of analysis which are based on interpreting internal kernel structures or trying to detect and recover hidden data [2].

Secondly, swap areas can be encrypted. It means that creating bit-by-bit copy of swap space partition from hard disk which was removed from compromised machine is useless.

Useful links:

Monday, August 21, 2006

“Memory forensics” related debugger extension DLLs for Microsoft Debuggers

One of the biggest problem with Windows “memory forensics” related tools is that such tools have to be updated concurrently because of new version of operating system or service pack. I’m thinking about offsets to fields inside various internal kernel structures which can vary. It is obvious that sooner or later you will have to use description of some internal structures to find digital evidence. If you write your own script or tool to parse a physical memory image you will have to take into consideration this problem. Even if you prepare signatures to grep some objects you will need information about offsets. Now just think about generic solution which is based on using symbols which can be download automatically or manually from Microsoft servers. Instead of hard coding information about Windows internal kernel structures and offsets for various versions of operating systems you just write one code for all of them. Firstly, your code is smaller (you can avoid many mistakes, too). Secondly, you can save a lot of time. So if you are lazy this solutions will be exactly for you :).

I decided to take a look closer at Microsoft Debugging Tools for Windows and debugger extensions which can be used by MS Debuggers and allow to use new debugger commands.

As we should know physical memory device objects (\\.\PhysicalMemory and \\.\DebugMemory) in Windows operating systems represent a raw data. It is impossible to load an image of such device object into WinDbg or KD because this file will be not recognized by Debuggers. Fortunately, you can convert raw data to recognizable format (crashdump format). A part of dump header format is described by Andreas Schuster at [1].

The next step is to download Debugging Tools for Windows and Symbols from [2]. If you have a direct access to the Internet you will not have to download Symbols because the debugger tool downloads Symbols for you automatically.

Debugger extension commands are exposed by DLLs. A short description about how to write new debugger commands can be found in the debugger.chm file which is installed with Debugging Tools for Windows (You have to use custom installation to install examples).

An environment described above can be used to perform offline analysis of physical memory dumps. In the other hand debugger extension DLLs can be used to verify system integrity on a live system. You have to use the livekd tool from [3] to load and execute commands exported by dll extensions.

A few useful functions which allow you to resolve Symbols or find out the offset are described below.
  1. GetOffsetByName(Symbol, Address), where the Symbol is the name of symbol like PsInitialSystemProcess or PsLoadedModuleList. The address of requested symbol is returned by the Address parameter.
  2. GetSymbolTypeId(Symbol, TypeId, Module), where the TypeId is an index within PDB file which is associated with the Module.
After that you can call other functions which take the TypeId and Module as parameters:
  • GetTypeSize(Module, TypeId, Size) to receive size of requested internal structure,
  • GetFieldName(Module, TypeId, Iteration, Name, MAX_PATH, NULL) to receive the Name of field pointed by a number = the Iteration,
  • or GetFieldOffset(Module, TypeId, Name, Offset) to receive the offset of requested field defined by the Name.

I wrote the function “offset(structname, fieldname)” which receives the offset to requested field of requested structure.

ULONG offset(CHAR *structname, CHAR *fieldname)

ULONG64 Module;
ULONG i1, TypeId;

g_ExtSymbols->GetSymbolTypeId(structname, &TypeId, &Module);

for (i1=0; ;i1++) {
ULONG Offset=0;

Hr1 = g_ExtSymbols->GetFieldName(Module, TypeId, i1, Name, MAX_PATH, NULL);
if (Hr1 == S_OK) {
g_ExtSymbols->GetFieldOffset(Module, TypeId, Name, &Offset);
if (strcmp(Name,fieldname) == 0) {
return Offset;
if (Hr1 == E_INVALIDARG) {
else {
dprintf("GetFieldName Failed %lx\n", Hr1);
return 0;

You can call this function in the following way:

ULONG OffsetAPL = offset("_EPROCESS","ActiveProcessLinks");

Of course it is enough to use “dt _EPROCESS” command to receive the same result so now something more useful. At my website [4] you can find the extension dll called hidden.dll which allows to detect all hidden processes – even hidden by the DKOM method. The command to call proper function is “!"full path to directory with hidden.dll file"hidden. allprocesses”. For example: “kd>!c:\temp\hidden.allprocesses”

You can record in external file all executed commands and results by using the command “.logappend “c:\forensics.log””.

As I mention above dll extensions can be executed at any version of Windows operating system. On a live system you have to use the livekd tool [3].

Useful links: