Handles are currently hacked in. Add a handle manager that allows something like this:

def ZwCreateFile_syscall(...):
    handle_data = (...)
    hFile = dp.handles.new(data)
    return hFile

def ZwReadFile_syscall(hFile: HANDLE, ...):
    handle_data = dp.handles.get(hFile, None)
    return

def ZwCloseHandle(hFile):
    dp.handles.close(hFile)

It should also support duplication (every handle points to refcounted data).

This is the first pass of the memory manager, I have not implemented it into dumpulator code. On its own, it works, it is not optimized at all.

Please let me know if you think anything should be done another way and how I should start replacing mem_map calls.

I have noticed if you do a dump on x64 OS of a x86 process (from task manager or procdump64 for example) dumpulator would try to emulate everything in x64 even though most of the dump is actually x86 causing some unexpected behavior, it doesn't happen if you do a x86 dump from x32dbg or from procdump(32).

I rewrote the memory manager to use the paging, although an issue that arose was that the checking function is a bit limited at the moment.

I have had my hand at trying to improve it - it will detect if something was allocating in already allocated space, or "enveloping" allocated space, but if it were to have one 'foot' in at any side of the region - it won't catch the allocation error.

This one is way faster ( mainly because the checker function had to be limited ), hopefully you guys can point me in the right direction if there is something glaringly wrong. Thanks.

Latest update to the handle manager broke how normal files were loaded and used. This is intended to fix this issue as well as adding in new functionality.

Inspired by the mapped files idea, I have implemented create_file within handles, this will depending on creation flags either open an existing file and load it in as a mapped file with it's data or create a new empty mapped file.

It's not perfect yet but it's a good start in the right direction, this way when emulating something that relies on touching files we should in theory be able to follow and capture the changes without them effecting the system, I'm sure there will be bugs behind this, something like \\.\PhysicalDrive0 might be an issue but I hope it would just raise an exception rather than try to load the entire disk into memory...

Added handles.create_file to add new files to mapped files Refactored FileObject and added write Added CreateDisposition flags Updated HandleTest exe

When unicorn gets an exception it needs to build a stack frame and set EIP/RIP on KiUserExceptionDispatcher

http://www.nynaeve.net/?p=201

It also needs a test application and a prepared dump in Windows Sandbox for CI.

First pass of a Module Manager for #17 Probably good for a PR as is but I would like to finish it and have good unit test coverage before then. Just creating the PR so you can have a look 👍

Resolves export forwards.

1. In Module's _parse_pe, if export points to .rdata store as an export forwarder (only caveat is x64 ntdll.dll's RtlNtdllName which is located in .rdata)
2. After all modules have been loaded in iterate over modules to then parse the export forwarders, this is done by overwriting the current export address with the resolved forward export's address
3. If we could not resolve the forwarder leave address to zero and assert we could not resolve the function within find_export

Currently does not resolve API sets, but we can cross that bridge later.

When a syscall is processes an enum with a 64bit dump the value is treated as a 64bit integer. Within Windows syscalls most enums are treated as 32bit values. I can't find the documentation for this but if you start going through syscalls that use enums you will notice the trend.

When a syscall with an enum is called Dumpulator processes it as the following resulting in a ValueError

Error: ValueError: 549755813892 is not a valid FSINFOCLASS

549755813892 = 0x8000000004

Previous argument in the call had the value: 0x8

I have tried to implement a CTypes style of python enum's to no avail. The following hotfix is enough to fix the issue, but could run into issues if an enum size is anything other than 32bit wide, however I have yet to find one that isn't 32bit wide.

elif issubclass(argtype, Enum):
    try:
        argvalue = argtype(dp.args[i] & 0xFFFFFFFF)
    except KeyError as x:
        raise Exception(f"Unknown enum value {dp.args[i]} for {type(argtype)}")

PR for #16 for the implantation of a handle manager.

Also added needed syscall functionality to get simple file reads working. Source I tested is included with the PR. This currently is working for 32bit.

Unicorn failed on 64bit due to the extended instruction set usage within ntdll!memset

SizeOfImage is not consecutive in memory. So unicorn fails to read the memory. Fix would be to read the memory in chunks of 0x1000 pages, or only pass the first page to pefile and load in pages when necessary.

Feature Request: Please look into addressing the issue where large dump files aren't easily processed by dumpulator.

Description: Dumpulator is taking an excessively long time to load a dump file of 40MB. I was testing the decryption of an RC4 encrypted configuration block from gh0stRAT. I dumped out the process and it was 40MB. I fed this dump into dumpulator and it was still trying to load after 20 minutes.

Sample: https://www.virustotal.com/gui/file/0a9c881b857d4044cb6dfeba682190d7d9dc6ef94bc149cac77f3e0f65e9c69a

Currently the "tests" are just running the "getting-started" examples. This doesn't scale and makes it difficult to find regressions.

The test framework:

• [x] A single unified Visual Studio solution with test executables
• [ ] Build the solution with GitHub Actions so new feature PRs can be tested in the same PR
• [x] A dump of a minimal Windows usermode environment (ntdll, kernel32, kernelbase, x64+x86)
• [x] A dump of a more "complete" environment (winsock, cryptapi, advapi32, shell32, user32, etc.)
• [x] Load the test executables with dp.map_module
• [x] Execute the (exported) test functions on a fresh Dumpulator instance each time
• [ ] Calculate code coverage per test

Necessary tests:

All of these should use /NODEFAULTLIB to not have to care about the MSVC runtime initialization (which uses a lot of unimplemented syscalls)

• [ ] ExceptionTest (for testing different exception scenarios)
• [ ] LoadLibrary (to test map_module and the whole PEB loading chain)
• [ ] StringEncryptionFun
• [ ] More depending on coverage...

The idea would be to generate a unique number for each syscall invocation and print it in the log. The sequence should be deterministic for a given dumpulator version (although preferably across versions as well). Uses cases:

• "Breakpoint" (stop emulation or literally a debugger breakpoint) at a certain ID you saw previously in the log for closer inspection.
• Enable single step tracing (slow) only after/between trace points.
• Dump the state after a certain trace point to avoid re-running the same code over and over again (needs #13)

Right now the state after a .call or .start persists. It would be nice to have a feature to roll back memory changes without having to reload the dump.

I think unicorn has a feature for this.

Currently the type system for syscalls is very rough and you need to do a lot of manual work. A type system similar to ctypes needs to be implemented where you can set struct members, work with enums etc.

Once the type system is complete a pdb/header parser can be implemented to support all the native types.