SuperAPI AI Directory

re-ida

Coding & development

View repository

Updated 10-15 21:34

About this skill

IDA reverse engineering workflow

When to use / when not to use

  • Use: you already have an IDA license/free version, and the target requires FLIRT library identification, Hex-Rays decompilation, or idapython batch processing
  • Use: large-scale repetitive annotation/export (headless batch processing idat -A -S, for the sequence see [[commands]])
  • Don't use: when the free version can't handle it (no Hex-Rays → use [[re-ghidra]] as a fallback); when you only need quick conclusions ([[re-triage]])
  • Don't use: in memory-constrained environments (large binaries lag, switch to [[re-radare2]]); for pure command-line quick analysis ([[re-radare2]] is lighter)
  • Don't use: for dynamic debugging of macOS/Linux targets ([[re-lldb]] / [[re-gdb]]); the IDA debugger is only most valuable for local Windows targets

Tool preparation

See [[re-analyze/platform-tips]]—static analysis is exempt from the sandbox; when using the IDA debugger for dynamic debugging, execute in the sandbox by default per the highest principle.

IDA (commercial version / free version)

  • Download: hex-rays.com (IDA Free, free version, supports x86/x64; other architectures and analyzers require the commercial version)
  • Windows: run the installer directly;choco install ida-free(community package, or manual download from the official website)
  • macOS/Linux: extract and run the tar package from the official website ida(Windows equivalent ida.exe, for headless use idat/idat.exe)
  • Verify: after startup, successfully open a sample and complete auto-analysis, the functions window has content;idat --help has output; in headless mode try running idat -A -L"t.log" /bin/true it exits normally
  • Version differences: since 7.5 the database is unified as .i64(old .idb/.idb64 can be upgraded); 8.x requires a 64-bit host; since 9.0, 32/64-bit installations are merged, with a built-in FLIRT signature manager and the idalib headless API——9.x executables no longer carry the 64 suffix(ida/idat one binary handles both 32/64-bit), only IDA ≤8.x separates ida/ida64、idat/idat64——see [[gotchas]] for details

idapython (built-in)

  • IDA 6.8+ has a built-in Python environment, no separate installation needed; IDA 9 uses idapyswitch(IDA root directory) to switch to an external Python (3.8–3.13)
  • Verify: menu File > Script command(Shift+F2) execute print(idaapi.IDA_SDK_VERSION) outputs the version

FLIRT signature libraries

  • IDA ships with the sig/ directory (flair); after applying, library functions are automatically annotated; IDA 9's signature manager can download more signatures online
  • Verify: after applying signatures to a known libc program,_init/malloc etc. are annotated as library functions

Procedure

  1. Import and auto-analysis:

    • File > New select the sample → wait for the bottom-left AU: analyzing to finish (no AU text and the analysis log stops)
    • Confirm Options > General > Analysis Auto-analysis is enabled; confirm segments and entry point are recognized
    • Headless batch processing: idat -A -S"myscript.py" -L"log.txt" sample(-A automatic mode, on Windows use idat.exe)
    • headless script starts with ida_auto.auto_wait() wait for analysis to complete, and ends with idc.qexit(0) exit——without these two lines you'll get an empty function list or the process will hang
  2. FLIRT identifies library functions:

    • File > Load file > FLIRT signature file... select the matching .sig(for MSVC choose mssdk、vc64rtf etc.; for GNU choose libstdc++ series)
    • After recognition, library function names are applied automatically; in the Functions window, library functions (light-colored) are separated from user functions — focus directly on non-library functions
    • Match failure (packed/obfuscated library) → indicates a packer or custom compilation, go to [[re-anti-analysis]]
  3. Shortcut mainline (daily static analysis):

    • x cross-references;n rename (functions/variables);y change type (e.g. int __cdecl f(int, char*));; comment (repeatable uses :)
    • G jump to address;Alt+T text search (string keywords);Alt+B binary search (byte patterns);Esc return to previous level
    • F5 Hex-Rays decompile;Tab switch between pseudocode/assembly views
    • For the complete shortcut and command table see [[commands]]
  4. Reading Hex-Rays decompilation:

    • F5 Enter pseudocode: first read the function signature and local variables, then follow the call chain (double-click a call to enter,Esc to return) to trace the data flow
    • Rename local variables/parameters to improve readability;y correct the calling convention and parameter types
    • When the pseudocode and disassembly don't match (structural differences caused by optimization), go back to the disassembly to verify — decompilation is a view, not fact
    • Algorithm/decryption recovery: use the decompiler output to locate loops, and use idapython to compute key constants (for the sequence see [[commands]] operation sequence 3)
  5. idapython batch processing (decryption loops/batch annotation):

    import ida_bytes, idc
    # 批量 patch: 0x401000 起 0x100 字节异或 0x55
    base = 0x401000
    for i in range(0x100):
        ida_bytes.patch_byte(base + i, idc.get_byte(base + i) ^ 0x55)
    
    # 批量重命名: 给所有未命名导入标注来源
    import ida_funcs, ida_name
    # 遍历函数: ida_funcs.get_func() / idc.get_func_name()
    

    Headless run: idat -A -S"script.py log.txt" sample, at the end of the script idc.qexit(0) to ensure exit (idc.qexit is the ida_pro.qexit alias, consistent in IDA 7.0+).

  6. Handling free version limitations:

    • The free version has no Hex-Rays: use disassembly + idapython (manually recover loops/algorithms as in step 5), or export directly to Ghidra: Menu File > Produce file > Dump typeinfo / use idb2pat to generate library signatures; or switch to [[re-ghidra]] for decompilation
    • Free version feature cuts: no local Hex-Rays (there is x86/x64 cloud decompilation you can try first), debugger limited to local x86/x64, no disassembler extensions — adjust the workflow according to the scope of the cuts
  7. Dynamic debugging (Windows local target, commercial version):

    • F9 run,F2 toggle breakpoint,F8/F7 single-step,Ctrl+F2 restart — same habits as [[re-x64dbg]]
    • Anti-debugging detection (IsDebuggerPresent etc.) locate statically first, then handle; for boundaries see [[gotchas]]
  8. Evidence verification (wrap-up): renames/comments/recovered structures are archived with the .i64 (File > Save, back up the original database with a copy); headless export artifacts (function list/decompiled .c) are filed for comparison against the [[re-triage]] initial survey values; key conclusions go into [[re-analyze/analysis-contract]] — annotations in IDA must be traceable back to the report, don't leave them only in the database

Function analysis context checklist

Before analyzing each function, first collect (see the [[re-analyze/analysis-contract]] context checklist): xrefs, strings referenced by the target function, caller/callee signatures, the named symbol table, recovered structs. Provide everything at once before analyzing; proactively request additional evidence with no more than 8 tool calls per function.

Deep analysis proceeds through the five steps of the "single-function deep analysis order" in [[re-analyze/analysis-contract]] (types → constants → vtables → identity → decompilation); under IDA, types use idapython get_type/struct definitions, and symbol/constant evidence can be exported with readelf / strings / objdump for assistance.

Cross-domain combinations

  • [[re-binary-core]]: alternative decompiler for workflow step 5 (RE_DECOMPILER=ida)
  • [[re-cracking]]: IDA is commonly used for locating registration algorithms/validation logic
  • [[re-malware]]: malicious sample debugging (IDA debugger) and analysis
  • [[re-plugin-dev]]: IDAPython is the plugin language, script → plugin engineering
  • When the free version has no Hex-Rays, interface with [[re-ghidra]]

Common pitfalls and traps

  • No decompilation in the free version: IDA Free has no Hex-Rays—confirm version capabilities first; if decompilation is missing, go straight to Ghidra instead of forcing your way through disassembly

  • Anti-debugging detection: samples can use IsDebuggerPresent/NtQueryInformationProcess to detect the IDA debugger (debugger default markers)—prioritize static analysis, or use hidden debugger plugins (ScyllaHide-like)

  • Large binaries lag: auto-analysis fully enabled is extremely slow for 100MB+ binaries—turn off some analysis options (such as constant propagation) or switch to [[re-radare2]]

  • FLIRT matches the real libraries "after unpacking"—for packed samples, unpack first, then run signatures

  • Address-type API return values are not checked for BADADDR: symptom—idapython scripts get 0xFFFFFFFFFFFFFFFF or similar addresses on some samples, and subsequent addressing/renaming is all wrong or crashes; cause—the IDA API uses BADADDR(-1) to indicate failure (name does not exist, invalid address, function does not exist), and the script continues using it without checking the return value; countermeasure—after calling every API that returns an address (get_name_ea/get_func , etc.), first check == BADADDR; on failure, print the scene first before continuing; scripts should first do read-only verification before writing to the database (see next item)

  • Modifying scripts run directly on the original database, and damage is hard to roll back: symptom—after a batch rename/byte-modify script finishes, a bunch of wrong annotations are found, and undoing is laborious or even irreversible; cause—skipping read-only verification and writing directly to the database, and also not operating on a copy; countermeasure—the official recommendation is to first run a read-only script (print database path/architecture/function and string statistics) to confirm expectations, and have modifying scripts verify incrementally step by step (minimal script → inspect output → add features); before batch changes, first copy the .i64/.idb or run on a copy

  • Hex-Rays interr (decompiler internal error): symptom—headless decompilation runs into a function and reports interr: create_stkvar(...) dtype=7 or similar, after which the whole process crashes, and skipping/retrying does not help; cause—an internal bug in the decompiler for a specific stack layout, unrelated to the sample/script; countermeasure—switch to Ghidra to complete that goal (cross-validation is also better), don't stubbornly fight the same function; for batch decompilation scenarios, first do a small trial run to confirm it is not triggered before going full-scale

  • Headless batch export did not wait for auto-analysis to complete (silently produces 0 functions): symptom—idat -A -S"export.py" sample batch export finishes, but the output has 0 functions or is severely incomplete, with no errors at all in the log; cause—in-A mode, script injection and auto-analysis run asynchronously in parallel, and the script iterates the function list before analysis completes, getting an empty set, with the failure silently swallowed; countermeasure—before collecting functions, first ida_auto.auto_wait() Block until analysis ends; the export loop uses try/except per function, so if a single function fails it is only recorded in a manifest (address/name/reason) and then continues; at the end summarize total/exported/failed without interrupting the full run; when exporting only non-library functions (FUNC_LIB flag), a small binary with no user code legitimately produces 0, verify with the dual channels of "log keywords + artifact counts" and tolerate this legitimate empty output; the call graph (callers/callees) is fully computed and cached first before entering the decompilation loop; export is ordered from cheap to expensive (strings → imports → exports → memory → decompile), with one file per function named by address (Source: LazyReverse (a0yami), MIT)

  • IDA 9 script migration (get_struc and others have been removed), version differences and anti-debugging boundaries see [[gotchas]]