From current README:

Currently uses multiple files for "database", each storing particular type of information. Switch to a single YAML file instead?

This tickets is for discussion of this issue, opinions welcome!

It seems that addresses beyond 0x80000000 are interpreted as negative numbers, even on a 64-bit python3. The address -2147360092 or -0x7ffe1d5c is actually 0x8001E2A4 which is a valid address:

./ScratchABit.py EXT0CB6Q.dec.P2.def --script import Loading CPU plugin arm_32_capstone Adding area: Area(0x0-0x1ffff, {'name': '.bin', 'access': 'RWX'}) Adding area: Area(0x808000-0x80ffff, {'name': '.bin', 'access': 'RWX'}) Adding area: Area(0x80000200-0x800301ff, {'name': '.bin', 'access': 'RWX'}) Loading EXT0CB6Q.dec.P21.frmw @0x0 Loading EXT0CB6Q.dec.P22.frmw @0x808000 Loading EXT0CB6Q.dec.P23.frmw @0x80000200 Processing section: entrypoints Performing initial analysis... 3000 Traceback (most recent call last): File "./ScratchABit.py", line 1030, in call_script(script) File "./ScratchABit.py", line 949, in call_script mod = import(script) File "/home/user/scratch/ScratchABit/import.py", line 2464, in SetRegEx(0x8000855A,"T",1,2) File "/home/user/scratch/ScratchABit/idc.py", line 65, in SetRegEx engine.analyze() File "/home/user/scratch/ScratchABit/scratchabit/engine.py", line 947, in analyze if not _processor.emu(): File "/home/user/scratch/ScratchABit/plugins/cpu/_any_capstone.py", line 176, in emu assert GetReg(op.addr, "T") == 0 File "/home/user/scratch/ScratchABit/idc.py", line 28, in GetReg if engine.ADDRESS_SPACE.get_flags(ea, engine.AddressSpace.ALT_CODE): File "/home/user/scratch/ScratchABit/scratchabit/engine.py", line 245, in get_flags raise InvalidAddrException(addr) scratchabit.defs.InvalidAddrException: (-2147360092, '-0x7ffe1d5c')

Hi,

I'm having trouble running ScratchABit on my machine:

$ python scratchabit.py example-elf
Traceback (most recent call last):
  File "scratchabit.py", line 721, in <module>
    load_target_file(loader, sys.argv[1])
  File "scratchabit.py", line 623, in load_target_file
    entry = loader.load(engine.ADDRESS_SPACE, fname)
  File "/home/user/esp/virtenv/ScratchABit/plugins/loader/elf.py", line 407, in load
    return load_sections(aspace, elffile)
  File "/home/user/esp/virtenv/ScratchABit/plugins/loader/elf.py", line 280, in load_sections
    if reloc["r_addend"] != 0:
  File "/home/user/esp/virtenv/ScratchABit/plugins/loader/pyelftools/elftools/elf/relocation.py", line 35, in __getitem__
    return self.entry[name]
  File "/home/user/esp/virtenv/ScratchABit/plugins/loader/pyelftools/elftools/construct/lib/container.py", line 35, in __getitem__
    return self.__dict__[name]
KeyError: 'r_addend'

Am I missing something? I use Ubuntu 14.04 with Python 3.4.3 Thanks

Hi Paul,

I just wanted to check if this is the recommended way to add a new plugin: http://www.esp8266.com/viewtopic.php?f=13&t=3756&p=22201#p22200

If you let me know the recommended way, I'll submit a PR with an addition to the README about adding plugins.

Cheers,

Angus

Saving (File->Save) seems to be slow on slow terminals, since it outputs the current location quite often. I would suggest to ouput the position less often to speed it up.

Hi, I'm writing a CPU plugin which calls ua_add_dref(0, xxx, dr_O) based on a value that's loaded into a register. Sometimes these values don't point into a valid address range, leading to a an error when I try to save the project:

2017-05-26 21:45:56,666 Exception processing user command
Traceback (most recent call last):
  File "/.../ScratchABit.py", line 123, in handle_input
    return super().handle_input(key)
  File "/.../ScratchABit/picotui/basewidget.py", line 69, in handle_input
    res = self.handle_key(inp)
  File "/.../ScratchABit/picotui/editor.py", line 208, in handle_key
    return self.handle_edit_key(key)
  File "/.../ScratchABit.py", line 543, in handle_edit_key
    saveload.save_state(project_dir)
  File "/.../ScratchABit/scratchabit/saveload.py", line 32, in save_state
    engine.ADDRESS_SPACE.save_addr_props(project_dir + "/project.aprops")
  File "/.../ScratchABit/scratchabit/engine.py", line 680, in save_addr_props
    fl = self.get_flags(addr)
  File "/.../ScratchABit/scratchabit/engine.py", line 247, in get_flags
    raise InvalidAddrException(addr)
scratchabit.engine.InvalidAddrException: (57259, '0xdfab')

Should I fix this in my plugin or should SAB handle this case and avoid adding the reference? What does IDA do?

[ An alternative or complement to my approach would be to create a data item with the loaded value, and let the engine add the reference automatically. AFAICS this feateure hasn't been implemented in SAB so far, though. ]

While working with ScratchABit I saved my database, during the save (I think) an unhandled exception occurred and scratchabit exited. My database doesn't open any more, the stack trace on opening is:

(Snip many lines of relocation information)
<Relocation (RELA): Container({'r_info_sym': 1, 'r_offset': 7, 'r_addend': 268, 'r_info_type': 11, 'r_info': 267})>
Loading state...
Traceback (most recent call last):
  File "/home/gus/scripts/scratchabit.py", line 587, in <module>
    load_state(project_dir)
  File "/home/gus/scripts/scratchabit.py", line 539, in load_state
    engine.ADDRESS_SPACE.load_funcs(f)
  File "/home/gus/dev/ScratchABit/engine.py", line 487, in load_funcs
    start, end = l.split()
ValueError: need more than 1 value to unpack

I tried renaming the *.bak files over the newer database files, but I get the same error.

Here are the relevant files: http://projectgus.com/misc/esp8266/xtensa_vectors.zip

Working from current git master 74323e9d2c4 and pfalcon/ida-xte[email protected] (one commit before current master in your fork).

There's a related bug to the database corruption bug, which is that the stack trace from the unhandled exception on save was printed "inside" ncurses, so it wasn't readable (got interleaved with the UI elements). Also following exit the console mode was set wrong, needed a reset.

Sorry I don't have time to investigate this further right now.

Continuing from #23 . See "orr/mov.w", "sub.w", "ldr" below.

│0802cf44 9878         ldrb     r0, [r3, #2]                                                                                           │
│0802cf46 5978         ldrb     r1, [r3, #1]                                                                                           │
│0802cf48 f268         ldr      r2, [r6, #12]                                                                                          │
│0802cf4a 41ea0021     orr/mov.w                                                                                                       │
│0802cf4e 0333         add      r3, #3                                                                                                 │
│0802cf50 0b44         add      r3, r1                                                                                                 │
│0802cf52 043a         sub      r2, #4                                                                                                 │
│0802cf54 7360         str      r3, [r6, #4]                                                                                           │
│0802cf56 f260         str      r2, [r6, #12]                                                                                          │
│0802cf58 fff732ba     b.w      0xa02c3c0                                                                                              │
│0802cf5c 5b10         asr      r3, r3, #1                                                                                             │
│0802cf5e 012b         cmp      r3, #1                                                                                                 │
│0802cf60 a5f10401     sub.w                                                                                                           │
│0802cf64 5ed0         beq      loc_0802d024                                                                                           │
│0802cf66 55f8042c     ldr                                  

Hi, this patch set is mostly cosmetics, but also one functional change: I implemented get_cmt/set_cmt so disassembler plugins can manipulate comments in the disassembly.

Please review, thanks.

Example of disasm:

│08020036          nlr_jump:                                                                                                           │
│08020036 0d4b         ldr      r3, [pc, #52]                                                                                          │
│08020038 1a68         ldr      r2, [r3, #0]                                                                                           │
│0802003a 002a         cmp      r2, #0                                                                                                 │
│0802003c 0df0de83     b.w      nlr_jump_fail                                                                                          │
│08020040 50           unk     0x50 ; 'P'                                                                                              │
│08020041 60           unk     0x60 ; '`'                                                                                              │
│08020042 10           unk     0x10                                                                                                    │
│08020043 68           unk     0x68 ; 'h'                                                                                              │
│08020044 18           unk     0x18                                                                                                    │
│08020045 60           unk     0x60 ; '`'                                                 

That b.w can't be right, should be conditional.

(Disassembling MicroPython's bare-arm port.)

File "scratchabit.py", line 506, in e.loop() "ScratchABit/pyedit/editor.py", line 254, in loop res = self.handle_key(key) File "scratchabit.py", line 314, in handle_key if '0' <= e.value[0] <= '9': IndexError: string index out of range

I have some firmware images dumped from the flash of a few IOT devices whose functionality I'm trying to re-implement using ESPHomeLib. They all appear to be in esp8266 v1 format. It appears that the esptool.py 'elf2image' command grabs various parts of the ELF binary (IROM, TEXT, etc) and packs them into the format that the on-chip bootloader wants. It does not appear to have any code to do the reverse - convert the flash image back to an ELF binary for analysis.

Is there any way I can analyze these files with ScratchABit, or is it pretty well dependent on having an ELF binary?

I've been wanting to look inside the BT/BLE controller stack at https://github.com/espressif/esp32-bt-lib, but ScratchABit doesn't currently handle SHN_COMMON sections.

Quoting the README:

See how to support other types of IDAPython plugins besides just processor modules.

Loader plugins would be highly appreciated. The problem is that Python loaders are hard to find. The existing loaders, covering widespread binary executable formats, are usually written in C++ so they need to be ported to IDAPython first. Fortunately, stand-alone Python libraries for parsing several common executable formats exist:

• pefile for parsing M$ PE files
• macholib for Mach-O executables

Creating basic loader plugins ontop of them should be easy. The existing pyelftools based ELF loader serves as a good example of how to do that. The first step is to enable support for loader plugins in ScratchABit...

PowerPC CPU is capable of running in either little-endian or big-endian modes. The required mode is usually specified in the executable itself. ELF container, for example, uses EI_DATA field of the ELF header for that purpose, see here.

The precise endianness mode must be set before disassembling.

I therefore propose to extend the CPU plugin interface with a possibility to specify the required endian mode as well as a way to retrieve endianness from container loaders.

BTW, it's not clear how IDA processor modules deal with this requirement. It looks like there is no consistent way to specify/retrieve endianness information in IDA, see this discussion.

With 2.0, Capstone-based ARM support went online, and Capstone supports several more architectures. Fairly speaking, ARM support enablement took a bunch of effort (and isn't really complete), but the cornerstone was supporting 2nd ISA for the code in the same address space. Beyond that, Capstone seems to over pretty weak semantic characterization of instructions, so bunch of that needs to be handled in arch-specific manner in the ScratchABit plugin.

Still, it shouldn't be a rocket science to enable more archs, and this ticket is submitted in the hope to find people who'd be interested to give it a try and share feedback.

References:

• https://github.com/pfalcon/ScratchABit/blob/master/plugins/cpu/_any_capstone.py (and git log -p --follow on it)
• https://github.com/pfalcon/ScratchABit/blob/master/Makefile.examples