SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsre-crypto-id

re-crypto-id

Security & auditing

View repository

Updated 10-15 21:35

About this skill

Encryption Algorithm Identification

When to use / When not to use

  • Use: when you have ciphertext (traffic or data blob) and are not sure what algorithm it is
  • Use: when the sample contains an encryption implementation and you need to narrow down the algorithm scope before decompiling
  • Use: when you suspect custom encryption (XOR/ROL/ROR transformations) rather than a standard algorithm
  • Don't use: when the algorithm is known (directly use [[re-crypto-keys]] to find the key, [[re-crypto-decrypt]] to decrypt)
  • Don't use: when standard library API calls are clearly visible (CryptEncrypt/OpenSSL symbols can be looked up directly—see [[re-crypto-keys]] import table clues)
  • Don't use: when purely static analysis can determine it is plaintext ([[re-triage]] low entropy/many readable strings)

Tool Preparation

Verify all tools before use. This skill focuses on static/offline analysis and can be done without a sandbox; the dynamic confirmation step (Frida) targets only already-running samples (sandbox by default, [[re-analyze/platform-tips]] highest principle).

python3 — fingerprint and entropy analysis scripts

  • For installation and verification see [[re-proto-rev]] Tool Preparation (python3)

binutils — strings/objdump to obtain constants and disassembly clues

  • Linux: apt install binutils / dnf install binutils / pacman -S binutils(mostly built-in)
  • macOS: brew install binutils(or the system's built-in otool as a substitute)
  • Windows/WSL: the Linux version inside WSL; on native Windows use Ghidra's bundled tools
  • Verify: strings --version；objdump --version

hexdump — hexadecimal view of ciphertext/constants

  • For installation and verification see [[re-fw-extract]] Tool Preparation (hexdump)

Detect It Easy (DIE) — optional, fast signature identification (commonly used on Windows)

  • Windows: download the portable version from GitHub releases https://github.com/horsicq/Detect-It-Easy（`diec.exe` CLI / die.exe GUI）；choco install die(available on some mirrors)
  • Linux: AUR yay -S detect-it-easy or the Linux version from releases
  • macOS: build from source (Qt dependency) or run the Windows version under Wine
  • Verify: diec --help output usage;diec sample.bin can output signatures

Procedure

Execute in order and record the result of each step. Pass the determined artifacts (algorithm hypothesis + evidence) to [[re-crypto-keys]] / [[re-crypto-decrypt]].

  1. Constant table fingerprints (AES S-box / CRC table / MD5 IV)：

    # 静态数据段找常量表候选：连续 256 字节、熵低、无 ASCII
strings -n 8 sample.bin | head -50
objdump -s -j .data sample.bin | head -60
# 搜索 AES S-box 开头（前 16 字节特征）
python3 - <<'EOF'
data = open('sample.bin','rb').read()
aes_sbox = bytes.fromhex('637c777bf26b6fc53001672bfed7ab76')
crc32_tab_be = bytes.fromhex('0000000077073096ee0e612c990951ba')   # 标准 CRC32 表前 16 字节（poly 0xEDB88320，显示序/BE）
crc32_tab_le = bytes.fromhex('00000000963007772c610eeeba510999')   # 同一表在 x86/ARM 小端二进制内存中的字节序
for name, sig in [('AES_SBOX', aes_sbox), ('CRC32_TAB(BE)', crc32_tab_be), ('CRC32_TAB(LE)', crc32_tab_le)]:
    i = data.find(sig)
    while i != -1:
        print(f"{name} @ 0x{i:x}"); i = data.find(sig, i+1)
EOF
    • Hit the AES S-box (256-byte table) → AES candidate; hit a CRC table → there is CRC/checksum (may pair with [[re-proto-rev]] step 3); hit the MD5 IV → MD5 candidate
    • Byte order: the hex listed above is in display order (BE reading order). In little-endian binaries (x86/ARM) the actual bytes of the constant table in memory are in LE order—for CRC tables also search 00000000963007772c610eeeba510999(already included in the script); MD5 IV display order is 67452301efcdab89..., LE serialization is 0123456789abcdeffedcba9876543210(search both modes)
    • No hit → does not rule out a dynamically generated table (see Pitfall 2), continue to the next step

  2. Entropy analysis to locate ciphertext (distinguish ciphertext from plaintext regions)：

    data = open('sample.bin','rb').read()
import math, collections
for base in range(0, len(data), 4096):
    blk = data[base:base+4096]
    if not blk: break
    c = collections.Counter(blk); n = len(blk)
    h = -sum((v/n)*math.log2(v/n) for v in c.values())
    if h > 7.0: print(f"0x{base:x}: entropy={h:.2f}  <- 高熵区(密文/压缩候选)")
    • High-entropy region (>7.0) → ciphertext or compressed data candidate, record the offset for [[re-crypto-decrypt]] to locate the input point
    • Low-entropy region but looks "messy" (no ASCII, no structure) → possibly custom encryption or a simple transformation (next step)

  3. XOR / ROL / ROR single-byte pattern detection：

    data = open('sample.bin','rb').read()
for key in range(256):
    dec = bytes(b ^ key for b in data)
    score = sum(1 for b in dec if 32 <= b < 127)
    if score > len(data) * 0.6: print(f"XOR key=0x{key:02x}, printable={score/len(data):.0%}")
    • Detection result "printable rate >60%" → single-byte XOR; hand off decryption to [[re-crypto-decrypt]]
    • ROL/ROR: observe the relationship between adjacent bytes of the ciphertext (x ^ rol(x) repeated occurrences for the same key); or look for a 256-entry rotation table (similar to an S-box but with values showing cyclic shift characteristics)
    • Cryptographic intuition also works: the result of a single-byte transformation usually preserves the original distribution characteristics, so try the simplest first and then escalate (see Pitfall 1)

  4. Common algorithm process characteristics (round count / block)：

    • Look for characteristic function forms in the disassembly/decompilation: AES has a 10/12/14-round (128/192/256-bit) loop structure + constant table references (together with Step 1); DES has 16 rounds + permutation tables (64-bit block); RC4 has a 256-byte KSA/PRGA loop
    • Data side:length can only assist in determining the mode, not distinguish the primitive. For modes like ECB/traditional CBC that pad to a whole block, the ciphertext is commonly an integer multiple of the block length; but CTR/OFB/CFB based on a block cipher can producearbitrary length equal to the plaintextciphertext (in NIST SP 800-38A the final block of OFB/CTR may be truncated to u bits, and the granularity of CFB is the segment size s); CBC-CTS can also avoid padding expansion. Therefore "arbitrary length"cannot be inferred RC4/XOR/ChaCha—for the same 37-byte plaintext,aes-128-ctr/ofb/cfb the ciphertexts are all 37 bytes,aes-128-cbc/ecb is the one that is 48 bytes
    • primitive identification relies onconstant tables / round function form / key schedule / IV-nonce-counter data flow: constant table fingerprints + round count (e.g. AES 10/12/14 rounds) determine the primitive; length and mode characteristics (whether padding is needed, whether the IV/nonce changes each time, whether the counter increments) determine the mode.AES block size is always 128 bit (16 bytes); 192/256 is the key size, there is no AES with a "24/32-byte block"
    • When a decompilation tool has auto-detection, use it first (Ghidra's FindCrypt script / IDA's FindCrypt2) for cross-confirmation
    • When a decompilation tool has auto-detection, use it first (Ghidra's FindCrypt script / IDA's FindCrypt2) for cross-confirmation

  5. Dynamic-side confirmation (Frida breakpoint at the encryption function)：

    • When the static conclusions are ambiguous (multiple candidates), run the sample inside the sandbox ([[re-sandbox]]) and use Frida to hook suspicious calls: 
      pip install frida-tools
frida -p <pid> -l hook.js

      // hook.js: 断在疑似加密函数，打印入参（密文/明文）与返回
Interceptor.attach(Module.findGlobalExportByName("crypt_fn"), {   // Frida 17+：旧写法 Module.findExportByName(null, ...) 已移除
  onEnter(args) { console.log("arg0:", hexdump(args[0])); },
  onLeave(ret)  { console.log("ret:", hexdump(ret)); }
});
    • Observe whether the input is high-entropy ciphertext (corresponding to the offset in Step 2), and whether the return becomes readable → confirm that this function is the encryption/decryption point
    • When the hook target name is uncertain, first frida -p <pid> -l /dev/stdin use Process.enumerateModules() to find dynamically loaded crypto libraries
    • Feed dynamic confirmation results back into static hypotheses: whichever candidate function actually receives the ciphertext is the one to use (see the algorithm combination in Pitfall 4: confirm the innermost layer first)

Cross-domain联合

  • [[re-protocol]]: Step 2 of this gateway workflow (encryption identification)—a required step when the traffic is ciphertext
  • [[re-malware]]: C2 communication encryption identification (re-malware Step 4: netcap → crypto-id → crypto-keys → crypto-decrypt)
  • [[re-firmware]]: algorithm identification for encrypted communication/encrypted firmware layers within firmware (together with the encryption-layer determination when [[re-fw-extract]] unpacking fails)
  • [[re-crypto-keys]] / [[re-crypto-decrypt]]: downstream—after identifying the algorithm, find the key and write the decryption
  • [[re-binary-core]]: decompilation corroboration (FindCrypt-type scripts for [[re-ghidra]] / [[re-ida]] / [[re-radare2]]); dynamic confirmation happens inside [[re-sandbox]]
  • [[re-anti-analysis]]: For packed samples, unpack first before doing constant-table fingerprinting (packer-layer constants will contaminate the fingerprint)

Common Pitfalls and Traps

  • For custom encryption, try simple modes (XOR) first before escalating: Symptom—spending half a day on AES fingerprinting only to find it's single-byte XOR; Cause—assuming a standard algorithm up front without doing cheap checks first; Countermeasure—the single-byte XOR/ROL/ROR detection in step 3 can be done within 30 seconds, then move on to constant-table fingerprinting and round-count analysis (cheap hypotheses first)
  • Table hiding (dynamic generation) → fingerprints fail: Symptom—no AES S-box/CRC table can be found in the static data section, leading to a misjudgment of "non-standard algorithm"; Cause—the algorithm dynamically generates the constant table at runtime (a common anti-analysis technique, see the [[re-anti-analysis]] domain); Countermeasure—dynamic confirmation in step 5: after running, search memory ([[re-memdump]]) for table signatures, or use Frida to break at the round function and see references
  • Algorithm combinations (XOR then AES) require layered identification: Symptom—the "plaintext" decrypted as AES is still garbage, or the XOR detection printable-rate is insufficient; Cause—multiple layers of encryption stacked, and the single-layer assumption is incomplete; Countermeasure—first peel the innermost/outermost layer (observe which layer, once removed, causes entropy to drop and readability to rise), confirm layer by layer, record each layer's identification result independently and then combine them (see the innermost-layer-first principle in step 5)
  • Mistaking compression for encryption: Symptom—a high-entropy region with entropy >7.0 is treated as encryption, and the decryption script does not match; Cause—zlib/LZMA compression is likewise high-entropy; Countermeasure—first check whether the first 2-4 bytes of the high-entropy region have a compression format identifier (the gzip header 1F 8B；78 9C is a common zlib CMF/FLG combination rather than gzip, and two bytes are insufficient as a sole criterion); if present, first use zlib.decompress/binwalk(see [[re-fw-extract]]) to try decompression before discussing encryption
  • Searching only for the S-box will miss variant implementations: Symptom—searching for the 256-byte S-box table yields no hit, leading to a misjudgment of "not AES" when it actually is AES; Cause—the implementation uses bitslicing/on-the-fly S-box computation (no table stored), but the key schedule often still retains the 16-byte Rcon table, or it switches to MixColumns multiplication tables (GF(2^8) multiply by 2/3/9/11/13/14); Countermeasure—additionally search for the Rcon sequence (01 02 04 08 10 20 40 80 1B 36 ..., 0x1B is the characteristic value) and the multiplication table layout, cross-confirming across multiple tables before classifying
  • A fingerprint hit ≠ the encryption function is in use: Symptom—finding an AES S-box/CRC table leads to analyzing it as that algorithm for a long time when the actual business logic uses different encryption; Cause—the constant table may come from uncalled static library code or packer-layer constants (unpack first, then fingerprint, see [[re-anti-analysis]]); Countermeasure—after a fingerprint hit, you must xref to confirm the table is referenced (who references it, and whether it is on the encryption path), and cross-check with round count/constant table layout/IV-nonce data flow (Do not use ciphertext length alignment as a primitive criterion), with final confirmation on the dynamic side (step 5)
  • Common signature pattern quick reference: Symptom—signature algorithm identification is slow; Cause—signature algorithms have fixed pattern families; Countermeasure—quickly compare by pattern: HmacSHA256(sorted_params, key) is the most common; MD5(params + salt + timestamp) is found in older systems; AES(JSON.stringify(params), key) is encryption rather than signing; RSA sign is rare (mostly in finance) (Source: reverse-skill field-journal, MIT)