re-anti-analysis

Anti-analysis adversarial (packer identification / unpacking / deobfuscation)

Complete workflow

1. Packer identification: [[re-packer-id]] (capability:unpack） — identify before acting. DIE/PEiD signature database scan, abnormal section names (UPX0/.aspack/custom), whether entry point points to a non-first section, entropy >7, minimal import table.First record OEP clues(packer entry features such as pushad near the EP), the identified packer name determines the path
2. Dispatch: simple compression packers (UPX/ASPack/FSG) → [[re-unpack-simple]] (capability:unpack); strong/virtualization packers (VMProtect/Themida) → [[re-unpack-advanced]] (capability:unpack）；Cannot identify → manual process(follow [[re-unpack-simple]] (capability:unpack)'s ESP law + memory breakpoints to manually find the OEP, or handle via the strong packer process), do not guess the packer name blindly
3. Simple unpacking: [[re-unpack-simple]] (capability:unpack） — prefer official/automatic unpacking (upx -d) → ESP law / memory breakpoints to find OEP → determine dump timing by plaintext/code materialization(single-layer packers are often near the real OEP, OEP is not a universal criterion; for timing see [[re-analyze/platform-tips]] key experience, dump-first by default) → IAT repair (Scylla/ImpREC, Windows)
4. Advanced unpacking: [[re-unpack-advanced]] (capability:unpack） — anti-debugging countermeasures (scyllaHide approach: NtQueryInformationProcess/timing differences) → combined stack backtrace/memory breakpoints to find OEP → dump (dump-first by default) → IAT repair (including redirection) → virtualized code region annotation (mark for bypass rather than restore)
5. Deobfuscation: [[re-deobfuscate]] (capability:deobfuscation） — if after unpacking there are still junk instructions / control flow flattening / string encryption: junk instruction removal, flattening restoration (D-810/manual), string decryption loop location and emulation, batch scripting, before/after comparison verification
6. Verification: archive unpacked artifact sha256 → re-run in sandbox ([[re-sandbox]] (capability:sandbox-setup) to determine it is cleanly unpacked, [[re-analyze/platform-tips]] highest principle) → import into [[re-ghidra]] (capability:decompilation、debugging) / [[re-ida]] (capability:decompilation、debugging) to confirm the OEP decompiles normally; only when the import table is resolvable is it complete. Return the artifact to the original calling domain to continue (malware samples back to [[re-malware]] (capability:malware-behavior、document-malware、evasion-analysis、key-extraction、threat-intel) behavior analysis, cracking targets to [[re-cracking]] (capability:license-analysis) authorization location)

Archive each step's result (evidence path + sha256, see [[re-triage]] (capability:triage)); packer fingerprints / unpacked artifacts are the source of [[re-ioc]] (capability:threat-intel) YARA features.

Anti-debugging methodology (common to the whole library, see [[re-analyze/analysis-contract]] index)

The anti-debugging analysis methodology applicable to this gateway (user practical experience), referenced by each skill's pitfalls and traps.

Basic detection

• AD1 exits immediately on startup, check debugger detection first: EP/TLS/initialization function
• AD2 IsDebuggerPresent does not mean complete anti-debugging: combined detection, do not patch just one point
• AD3 PEB BeingDebugged user-mode quick detection: look for PEB access/BeingDebugged/NtGlobalFlag
• AD4 NtGlobalFlag anomaly → check heap debug flags
• AD5 CheckRemoteDebuggerPresent unreliable: may be hooked/replaced/underlying
• AD6 NtQueryInformationProcess is the key point：ProcessDebugPort/ProcessDebugObjectHandle/ProcessDebugFlags
• AD24 WinAPI breakpoints not hit → may go directly through Native API: the program does not necessarily call high-level APIs like IsDebuggerPresent; it may directly call NtQueryInformationProcess, NtSetInformationThread, the Zw series. When analyzing, trace downward.

Exception mechanisms

• AD7 Exception does not crash → possibly anti-debugging: INT3/INT 2D/single-step exception
• AD32 INT3 has more than one form: CC vs CD 03 long breakpoint, exception behavior differs
• AD33 INT 2D is a special Windows anti-debugging point: behavior differs under a debugger vs normal execution
• AD34 ICE instruction detects single-step：F1 ICEBP
• AD26 Abnormal exception breakpoint behavior → check VEH: AddVectoredExceptionHandler—INT3 is actively handled/single-step exception is swallowed/execution continues after the exception
• AD27 SEH chain exception → possibly anti-debugging control flow: control jumps/hidden flow/detect debugger

Timing detection

• AD8 Behavior changes after breakpoint → check Trap Flag: EFLAGS/exception count
• AD9 Time suddenly slows down → check timing anti-debugging：QPC/GetTickCount/timeGetTime/RDTSC
• AD10 RDTSC detection: do not look at only one read: start-end delta, comparison threshold, branch location
• AD11 Sleep detection → check time acceleration: Sleep(5000)+check actual elapsed time—debugging environment/sandbox acceleration/time jump
• AD35 Timing detection is not only RDTSC：RDTSCP/QPC/GetTickCount64/APIC timer

Environment and tool characteristics

• AD12 Window/mouse detection → environment authenticity: automated sandbox
• AD16 Debugger window detection is low-level but common: OllyDbg/x64dbg/WinDbg/IDA window class or process name
• AD17 Process list detection → check tool fingerprints: debugger/monitor/sandbox, prone to false positives
• AD38 Window detection is only a low-level means: advanced samples tend more toward API behavior/memory characteristics/abnormal behavior
• AD39 Parent process detection: explorer→app vs debugger→app launch chain
• AD40 Command line detection: debugger/sandbox/analysis environment parameters
• AD31 Program only crashes in x64dbg → check debugger characteristics: x64dbg module/window class/DLL name/memory characteristics
• AD36 CPU characteristic detection may be used for anti-debugging: the program may read CPUID, MSR, TSC features to distinguish a real machine, VM, or debugging environment.

Hardware traces

• AD29 Hardware breakpoints also fail → may detect DR registers: DR0-DR3/DR7 or abnormal behavior
• AD43 Debug registers are not the only hardware traces: DR/LBR/performance counter state
• AD15 Hardware breakpoint detection → check DR registers：DR0-DR3/DR6/DR7

Threads and debug objects

• AD25 Thread suddenly disappears → check NtSetInformationThread: ThreadHideFromDebugger—main thread normal/logic breakpoints ineffective/debugger cannot see exceptions
• AD30 After the debugger pauses, logic changes → check Debug Object：ProcessDebugObjectHandle
• AD37 The debugger causes thread scheduling changes: thread execution order/lock contention/APC scheduling

Self-modification and breakpoints

• AD13 Breakpoint is cleared → check hardware breakpoints/self-check: 0xCC detection——verify code segment/compare hash/scan for 0xCC
• AD14 Code is restored to its original state after execution → possibly self-modifying: static content ≠ actual executed content, requires memory dump/execution tracing
• AD28 Code changes after a breakpoint → check self-modifying code: 55 8B EC→CC 8B EC——verify code/restore bytes/actively detect breakpoints

Deep detection

• AD41 PEB detection is not just BeingDebugged：NtGlobalFlag/ProcessHeap flags/HeapForceFlags
• AD42 Kernel debug detection：KdDebuggerEnabled/KdDebuggerNotPresent
• AD44 Anti-debugging may deliberately create fake logic: return wrong data/modify algorithm results/delayed execution/skip key logic——more common than exiting
• AD21 Kernel-level anti-debugging → user-mode patch is ineffective: driver/kernel callback/kernel object inspection requires analyzing the kernel path
• AD22 Anti-virtualization ≠ anti-debugging: different targets——observer vs analysis environment, do not confuse the two

Methodology

• AD18 TLS Callback executes earlier than Entry Point: entry breakpoint did not trigger anti-debugging → check TLS
• AD23 Entry breakpoint not triggered → not necessarily not executed, may have been preempted by TLS callback: A Windows PE's TLS callback can execute code before the Entry Point. When encountering a program that exits immediately on startup and the entry breakpoint is ineffective, check the TLS Directory first. Typical scenarios: anti-debug initialization, packer initialization, environment detection.
• AD19 Don't rush to patch anti-debugging logic: first confirm how the detection result affects control flow——exit/reduced functionality/fake data/delay
• AD20 Multiple anti-debugging points → find the convergence point: multiple detections converge on one flag/state variable/error handling function
• AD45 Patching the anti-debugging point may trigger integrity checks: JNZ→JMP/CALL→NOP triggers CRC/hash/self-check, find the detection chain first

Anti-virtualization methodology (common to the whole library, see the [[re-analyze/analysis-contract]] index)

The anti-virtualization analysis methodology applicable to this gateway (user practical experience), referenced by each skill's pitfalls and traps.

Instruction-level and firmware information detection

• AV-001 CPUID hypervisor bit detection
  • Trigger: the program executes CPUID leaf 1 and checks ECX bit 31 (hypervisor present bit)
  • Observation: ECX bit 31 is set, indicating it is currently running under a hypervisor
  • Inference: the sample is probing the virtualization environment——the most common starting point for anti-virtualization detection
  • Action: continue with the CPUID vendor leaf (0x40000000) to confirm the hypervisor type (Microsoft Hv / VMwareVMware / KVMKVMKVM, etc.)
  • Constraint: hypervisor_present ≠ necessarily a malicious analysis environment——cloud hosts and normal user systems also have a hypervisor; this bit alone cannot determine that it is an analysis environment
• AV-002 Registry BIOS/System information
  • Trigger: the program reads registry BIOS/System information (Manufacturer / ProductName / BIOS Version)
  • Observation: the read values are used to correlate with VMware / VirtualBox / Hyper-V signature strings
  • Inference: Registry firmware information is a common source of VM fingerprinting
  • Action: Locate the read points and the signature string matching logic, and collect all matched VM signature items
  • Constraint: A single string match cannot be used as the basis for a VM determination—multi-signature joint determination is required

Time and Hardware Characteristics

• AV-003 Time Difference Before and After Sleep
  • Trigger: The measured time difference before and after Sleep is significantly shorter than expected
  • Observation: The sample checks the actual elapsed time after Sleep(N) and finds it significantly less than N—time is accelerated/compressed
  • Inference: The sample may be detecting sandbox time acceleration/virtualized time (timing APIs are hooked or the clock is accelerated)
  • Action: Trace the sources of RDTSC / QPC / GetTickCount—locate the time reading points and the difference calculation logic, and confirm the clock source used by the sample
  • Constraint: Time detection often combines multiple points (RDTSC with CPUID forcing a VM exit, GetTickCount / QPC cross-validation), so do not handle only one place
• AV-004 Hardware Device Enumeration
  • Trigger: The program enumerates hardware devices (PCI / Disk model / GPU name)
  • Observation: The enumeration results are compared against virtual machine hardware model characteristics
  • Inference: VM fingerprinting—hardware device models are a common source of VM fingerprints
  • Action: Build a hardware characteristic set rather than making a single-point determination—list all checked hardware items and evaluate the set as a whole
  • Constraint: A single hardware characteristic (such as a certain disk model) is insufficient to determine a VM; joint evaluation of the characteristic set is required

Environment Scoring and Interaction Detection

• AV-005 Detection Results Converge into a Unified Variable
  • Trigger: The detection results of multiple VM detection functions are written into a unified variable (is_vm / sandbox_score / env_flag)
  • Observation: Multiple detection code paths converge into the same variable/state
  • Inference: The sample uses an environment scoring mechanism—multiple detection items are counted/weighted into an environment score
  • Action: Trace the convergence variable—set breakpoints at variable reads/writes or statically trace xrefs, avoiding analysis detection point by detection point
  • Constraint: Looking at only a single detection point will miss the overall scoring logic—the scoring threshold and combination method are key
• AV-006 User Interaction Detection
  • Trigger: The program detects user interaction (mouse / keyboard / uptime / window activity)
  • Observation: The sample checks interaction traces such as mouse movement, keyboard input, system uptime, and window activity
  • Inference: The sample attempts to distinguish a real user from an automated sandbox—interaction traces are a behavioral-level environment fingerprint
  • Action: Shift to behavioral-level analysis rather than hardware fingerprints—interaction-related branches require dynamic cooperation (simulated interaction or observation in a real environment)
  • Constraint: Interaction detection is mostly behavioral-level and difficult to fully reconstruct purely statically—it needs to be combined with dynamic observation

Virtualization Channels and Execution Paths

• AV-007 VMware Proprietary Channel Calls
  • Trigger: The program calls the VMware backdoor / Guest Additions / vmtools
  • Observation: The sample accesses VM proprietary channels (VMware I/O port backdoor, vmtools service/process, Guest Additions module)
  • Inference: High-probability virtualization detection—these channels truly exist only in a virtual machine environment
  • Action: Confirm how the call results affect control flow—locate the backdoor call point and the locations where the return value is used, then decide on the patch strategy
  • Constraint: Detecting a VM channel does not necessarily mean it exits—look at where the detection result flows before taking action
• AV-008 Detection Results Affect the Execution Path
  • Trigger: Detection result affects the execution path — the VM environment enters a different branch
  • Observation: Detection as "is VM" and "not VM" lead to different code branches
  • Inference: Anti-virtualization is not only used for exit — it may change sample behavior (fake logic / reduced functionality / different payload)
  • Action: Analyze the differences between the two branches — compare the code and data of the VM / non-VM paths; don't just patch the detection point
  • Constraint: Same principle as anti-debugging fake logic (see anti-debugging methodology AD44) — after patching, don't directly trust the output

When to use which atomic skill (selection tree)

Identify first, then choose the unpacking path; if it can't be identified, go through the manual process.

• Input is an unknown sample / suspected of being packed → [[re-packer-id]] (capability:unpack) identify → branch by result (don't skip steps)
  • Identified as a simple compression packer (UPX / ASPack / FSG etc.) → [[re-unpack-simple]] (capability:unpack）
  • Identified as a strong packer / virtualization packer (VMProtect / Themida etc.) → [[re-unpack-advanced]] (capability:unpack）
  • Packer name cannot be identified(no signature match) → manual process: first try the ESP law + memory breakpoint from [[re-unpack-simple]] (capability:unpack); if it fails or anti-debugging/virtualization features are found → [[re-unpack-advanced]] (capability:unpack) manual unpacking
• Goal is only to confirm the packer("what packer is this") → [[re-packer-id]] (capability:unpack) is enough; do not proceed to unpacking
• After unpacking there is still code obfuscation(junk instructions / flattening / string encryption) → [[re-deobfuscate]] (capability:deobfuscation）
• Target confirmed to be unpacked → this gateway is not needed; switch to [[re-binary-core]] (capability:decompilation、debugging、memory-dump、elf-parser、pe-parser、macho-parser; [[re-ghidra]] (capability:decompilation、debugging) / [[re-ida]] (capability:decompilation、debugging) / [[re-radare2]] (capability:decompilation)) for direct analysis
• Detection evasion/EDR countermeasures (AMSI/ETW bypass, fileless, lolbin chains, the sample being detected "why") → [[re-evasion]] (capability:evasion-analysis; dynamic first, with [[re-sandbox]] (capability:sandbox-setup) / [[re-memdump]] (capability:memory-dump））
• The whole unpacking process needs to read process memory → [[re-memdump]] (capability:memory-dump; dump by default, timing determined by materialization); Windows debugging → [[re-x64dbg]] (capability:debugging); Linux/Wine debugging → [[re-gdb]] (capability:debugging）

Cross-domain combination

• Malware packing: [[re-malware]] → this gateway (packer-id → unpack-*), rerun the unpacked product in the sandbox then perform behavioral analysis
• Cracking prerequisite: [[re-cracking]] → this gateway (if packed, unpack first, then locate the authorization logic), the authorization-locating product is used for patching/keygen
• Mobile hardening: [[re-mobile]] / [[re-apk]] → this gateway (Android hardening unpacking)
• Static discovery of packer: [[re-binary-core]] / [[re-format-pe]] / [[re-format-elf]] / [[re-imports]] (import table extremely small/packer hides imports) → switch to this gateway
• Dynamic assistance: [[re-gdb]] (break at OEP, unpack under Wine), [[re-x64dbg]] (Windows OEP + Scylla), [[re-memdump]] (dump by default, timing determined by materialization), [[re-tracing]] (used together when an anti-debugging sample detects the trace environment)
• Unpacked artifact verification must be sandboxed: [[re-sandbox]] ([[re-analyze/platform-tips]] highest principle)
• Packer constant contamination fingerprint: [[re-crypto-id]] For packed samples, unpack first before doing constant-table fingerprinting
• Detection-evasion countermeasures: [[re-evasion]] — packers/obfuscation are "static anti-analysis", while AMSI/ETW/fileless/lolbin are "detection countermeasures"; both belong to this gateway (dynamic analysis is mandatory [[re-sandbox]])
• This gateway is invoked by [[re-analyze]]'s triage "sample is packed / unpack" path (re-anti-analysis → packer-id → unpack-* → verify)

Common pitfalls and traps

• Skipping identification and unpacking directly: Symptom — grabbing a sample and jumping straight into the debugger/OEP workflow, running into strong packer anti-debugging or wasting time; Cause — not determining the packer category first; Countermeasure — first [[re-packer-id]] identify; simple packers auto-unpack in seconds, only strong packers are worth the manual workflow

• Dumping too early: Symptom — what gets dumped is still the packer's initial state (compressed/encrypted data), an empty unpack; Cause — dumping before the plaintext/code has materialized; Countermeasure — for single-layer packers, near the real OEP is usually appropriate, but OEP is not a universal criterion; for multi-stage/on-demand decryption/virtualization/reflective loading, etc., determine the dump point by "write→execute transition + memory protection changes + payload header/import recovery" ([[re-analyze/platform-tips]] key experience, dump first by default)

• Treating IAT without fixing it as a finished product: Symptom — after unpacking the import table is a mess, and Ghidra/IDA decompilation is total chaos; Cause — the import table is dynamically rebuilt inside the packer and the dump was not fixed up; Countermeasure — on Windows use Scylla/ImpREC to fix it, then re-decompile to verify after fixing

• Packed within packed: Symptom — after unpacking one layer you find another layer inside (e.g., Themida wrapped inside UPX); Cause — multi-layer packing is a common anti-analysis technique; Countermeasure — verify each layer as you unpack it (sha256 + import table + sandbox re-run); when layers are nested, return to step 1 and re-identify

• Anti-debugging interfering with the whole workflow: Symptom — debugger breakpoints do not take effect, the process exits, timing is disrupted; Cause — the sample detects the debugger (NtQueryInformationProcess, timing differences, debug port); Countermeasure — [[re-unpack-advanced]] anti-debugging countermeasures first (scyllaHide approach), attack the outermost detection first

• OutputDebugString timing trap: Symptom — the sample behaves differently after calling OutputDebugStringA/W , or under a debugger some code segment is noticeably slower and triggers a timing check; Cause — the debugger has a fixed slow path for handling ODS messages (receive-format-flush), and the sample uses the timing difference of a long-string ODS call as a debugger probe (still used by malicious samples in 2024-2025); Countermeasure — locate the ODS call site and break/bypass it (return a fake value or skip it directly), or use scyllaHide to hide ODS handling, and investigate together with timing-difference detection (AD9)

• High entropy ≠ encryption: Symptom —.of2 a section like this with 4KB entropy 7.9 is judged "fully encrypted" and statically abandoned; Cause — it may be compression (deflate without standard header), VMProtect mutated code, or pure data; Countermeasure —partition sampling(the 64B entropy at the entry region may be only 5.6 and is the real code); first rule out compression and mutation before talking about encryption; mutation code characteristics:ror/xor 链 + 跨区 jmp, sequential decoding produces fcom/int 88h/ret far -like junk but the entry 24 bytes are call/syscall structure

• Full debug-system chain (anti-debug points): Symptoms—not understanding "why attaching pauses the program," not knowing where to change things for anti-debugging; Cause—lack of full-chain understanding of the Windows debugging system; Countermeasure—chain:DebugActiveProcess→DbgUiConnectToDbg(create DEBUG_OBJECT and attach the debugger first, store the handle at TEB->DbgSsReserved[1](offset 0x16A8)) →NtDebugActiveProcess→DbgkpSetProcessDebugObject(the debuggee is attached) +DbgkpPostFake*Messages(the fake "thread created/dll loaded" messages that spam the screen on attach); debug event flow: the debuggee detects it is being debugged → creates a DEBUG_EVENT and links it into the EventList doubly linked list → sets the EventsPresent semaphore to wake the debugger → waits forever on ContinueEvent;Anti-debug point：DbgUiRemoteBreakIn create a remote thread to execute int3 to break the program (if you remove that int3, attaching cannot break in); TEB+0x16A8 is the debug object handle (detect/tamper to interfere with the debugger); exceptions are first sent to the debugger (DbgkSendApiMessage→DbgkpQueueMessage, flag=0 requires waiting vs NOWAIT fake message)

• so function-level encryption (dlsym runtime decryption): Symptoms—individual functions in the so (such as mymyadd) decompile as garbage/undefined instructions, while the rest are normal; Cause—so hardening encrypts the target function by symbol name: the function code section is ciphertext at rest, and at runtime before dlopen+dlsym it is first decrypted and overwritten, then re-encrypted after the call; Countermeasure—restoration approach: dlsym symbol resolution chain (__loader_dlsym→dlsym_impl→do_dlsym→dlsym_handle_lookup→find_symbol_by_name, choose between GNU hash / ELF hash: GNU hash requires computing gnu_hash + Bloom filter + finding the symbol table from PT_DYNAMIC), following ELF header → program header table → PT_DYNAMIC to locate the symbol table and hash table, and compute the function location and size; dynamic side: after dlopen/dlsym returns, dump at the decrypted function memory (timing of the dump is key experience); variant: per-section type XOR (MODE_DUPLEX) requires a custom linker to load and run; there is also encrypting .text as a whole and self-decrypting—first confirm whether it is function-level or section-level before choosing a solution