re-iot-proto

IoT protocol reverse engineering (MQTT / CoAP / BLE / Zigbee / NFC smart cards)

When to use / when not to use

• Use: when you have IoT device traffic (pcap / live capture) and need to parse MQTT / CoAP semantics
• Use: BLE device (adv / GATT) interaction analysis
• Use: Zigbee (802.15.4) network capture and ZCL command analysis
• Use: when you need to understand "device report / downlink control" semantics and tie into firmware
• Don't use: generic custom protocols (go to [[re-proto-rev]])
• Don't use: ordinary network traffic capture (go to [[re-netcap]])
• Don't use: encrypted traffic without keys—first [[re-crypto-id]] / [[re-crypto-keys]]

Tool preparation

For physical device capture, mind the authorization boundary (see step 5); dynamic scenarios default to sandbox / isolated network ([[re-analyze/platform-tips]] highest principle). Verify all tools before use.

wireshark / tshark — main parsing tool (MQTT/CoAP/BLE/Zigbee dissectors built in)

• Linux: apt install wireshark tshark / dnf install wireshark-cli wireshark / pacman -S wireshark-cli wireshark-qt(Arch has split it into wireshark-cli + wireshark-qt; when installing on Debian, select "allow non-root capture", or sudo dpkg-reconfigure wireshark-common then add the user to the wireshark group)
• macOS: brew install --cask wireshark(includes tshark CLI; or brew install wireshark CLI only)
• Windows: choco install wireshark; inside WSL use the Linux version
• Verify: tshark --version；tshark -G fields | grep -iE 'mqtt|coap|btatt|zbee' lists field names (field names are slightly adjusted between versions; use that output as authoritative)

scapy — optional parsing / construction

• All platforms: pip install scapy（Python 3）
• Verify: python3 -c "import scapy; print(scapy.__version__)"

MQTT — mosquitto client (active subscribe/publish verification)

• Debian/Ubuntu: apt install mosquitto-clients（mosquitto_pub / mosquitto_sub）
• Fedora: dnf install mosquitto；Arch: pacman -S mosquitto；macOS: brew install mosquitto；Windows: choco install mosquitto
• Verify: mosquitto_pub -h；mosquitto_sub -h
• GUI alternative: MQTT Explorer (cross-platform desktop)

CoAP —— coap-client / aiocoap

• Debian/Ubuntu: apt install libcoap3-bin(provides coap-client)
• Fedora: dnf install libcoap libcoap-utils(coap-client is in the libcoap-utils subpackage); Arch: pacman -S libcoap（extra）；macOS: brew install libcoap
• If the distro package does not include sample programs → pip install aiocoap(command aiocoap-client，Python 3）
• Verify: coap-client -h(or aiocoap-client -h）

BLE —— bluez + ubertooth

• bluez（btmon / hcitool）: Debian/Ubuntu apt install bluez；Fedora dnf install bluez；Arch pacman -S bluez bluez-utils; verify btmon --help
• ubertooth (2.4GHz sniffing hardware): Debian/Ubuntu apt install ubertooth(firmware installed separately ubertooth-firmware）；Fedora dnf install ubertooth(available in Fedora 43/44 official repositories); not in Arch official repositories → AUR; macOS brew install ubertooth; verify ubertooth-btle -s
• Android btsnoop (no installation needed): Developer options → check "Enable Bluetooth HCI snoop log" → reproduce the interaction → adb bugreport or pull /sdcard/btsnoop_hci.log(btsnoop format, opens directly in Wireshark)
• nRF Sniffer for BLE (Nordic, optional): requires an nRF-series dongle + Wireshark extcap (mainly Windows/macOS)

Zigbee — Wireshark + sniffer hardware + KillerBee

• Wireshark zbee dissector built in (zbee_nwk / zbee_aps / zbee_zcl); decryption in Preferences → Protocols → ZigBee → Security (check Decryption, fill in Network Key / Link Key)
• sniffer hardware (no distro package, hardware solution): TI CC2531 USB stick (flash Sniffer firmware) or Silicon Labs EFR32MG (flash sniffer with Simplicity Studio + Wireshark extcap)
• KillerBee (Python, ZCL capture / construction / replay): not available in Debian/Fedora/Arch official repositories → pip3 install killerbee(PyPI; depends on libusb/libpcap—Debian/Ubuntu apt install libusb-1.0-0-dev libpcap-dev、Fedora dnf install libusbx-devel libpcap-devel、Arch pacman -S libusb libpcap); firmware flashing sudo ./kb_install_firmware; verify kb --version
• ZBOSS protocol stack (DSR's Zigbee stack) documentation as NWK/APS/ZCL structure reference
• Key extraction linkage: [[re-firmware]] (read keys from firmware flash, together with zbgoodfind) / [[re-crypto-keys]]

Operating Steps

Execute in order, archive each step's artifacts (pcap / keys / parsing scripts + sha256).

1. Protocol Identification：

   tshark -r cap.pcap -q -z io,phs                                   # 协议分层统计
   tshark -r cap.pcap -T fields -e tcp.dstport -e udp.dstport | sort | uniq -c | sort -rn
   

   • Port quick reference: MQTT 1883 (plaintext) / 8883 (TLS); CoAP 5683 (UDP) / 5684 (DTLS); BLE: advertising and GATT (bthci_evt / bthci_acl / btatt layers, no ports); Zigbee: 802.15.4 frames (no ports, 2.4GHz channels 11-26)
   • Non-standard port → Wireshark Decode As / tshark -d tcp.port==9999,mqtt force specify
   • Feature confirmation: MQTT first byte is the control type (CONNECT 0x10 / PUBLISH 0x30 ...); CoAP header Ver field is 2 bits (value 01); BLE HCI event packet 0x3e; Zigbee frame first 2 bytes are the 802.15.4 frame control field
2. MQTT/CoAP parsing (topic / payload)：

   tshark -r cap.pcap -Y mqtt -T fields -e mqtt.topic -e mqtt.msg
   tshark -r cap.pcap -Y 'mqtt.topic contains "ota"' -T fields -e mqtt.msg    # 按主题过滤
   tshark -r cap.pcap -Y coap -T fields -e coap.code -e coap.uri_path -e coap.opt.content_format
   

   • topic is the control semantics: device/<id>/sensor、ota/update、cmd/relay etc.—first bucket by topic (-e mqtt.topic | sort | uniq -c）
   • payload is commonly JSON / protobuf / custom binary → field meanings correspond to the device App or firmware (step 5)
   • Active interaction verification (within authorization): mosquitto_sub -h <broker> -t '#' -v(subscribe to all),mosquitto_pub -h <broker> -t <topic> -m '{"relay":1}'；coap-client -m get coap://<ip>/sensor、coap-client -m put -e '{"on":1}' coap://<ip>/relay
   • Encryption (8883 / 5684): use SSLKEYLOGFILE to decrypt TLS; RSA private keys only apply to old-style static-RSA key exchange (TLS 1.3 and ECDHE sessions cannot be decrypted); for DTLS PSK, fill in the hexadecimal pre-shared key in the Wireshark dtls preferences (key sources see step 4 and [[re-crypto-keys]])
3. BLE capture (adv / GATT)：
   • Android btsnoop: enable snoop log in developer options → reproduce App-device interaction → pull btsnoop_hci.log and open with Wireshark (auto-decodes HCI)
   • Linux: btmon -w out.btsnoop(together with the target connection process; the written btsnoop format opens directly in Wireshark)
   • ubertooth (can sniff 2.4GHz when there is no host-side Bluetooth): ubertooth-btle -f -c 0(follow mode to observe advertising / connection packets),ubertooth-btle -s(scan)—real-time display only, no disk write; pair with on-screen analysis; to write to disk and open in Wireshark use btmon / Android btsnoop / nRF Sniffer (extcap)
   • GATT parsing:

     tshark -r ble.pcap -Y btatt -T fields -e btatt.opcode -e btatt.uuid -e btatt.value
     

   • Focus on: advertising packets (device name / manufacturer data / service UUID) and GATT read/write characteristics (sensor values, control commands, firmware version)
4. Zigbee（802.15.4）：
   • Hardware sniffer capture (CC2531 / EFR32, select the correct channel) → Wireshark zbee parsing
   • Decryption: Preferences → Protocols → ZigBee → Security: check Decryption, fill in Network Key (16-byte hex) and Link Key (the common default TC link key for Zigbee 3.0 is ZigBeeAlliance09）
   • Key acquisition paths: ① capture the network formation process (Trust Center distributes the network key to new nodes) ② default TC link key / install code ③ firmware flash extraction ([[re-firmware]] / [[re-hardware-io]], together with zbgoodfind to find keys in the firmware) ④ physical access to the coordinator to read configuration
   • Layer structure: NWK (addressing/encryption) → APS (endpoints/clusters) → ZCL (commands: on/off / read attributes / OTA)—tshark -r zigbee.pcap -Y zbee_zcl -T fields -e zbee_zcl.cmd to see commands (field names according to tshark -G fields | grep -i zbee )
   • KillerBee interaction (within authorization): zbdump(packet capture),zbstumbler(network scanning),zbreplay(replay ZCL commands)
5. Device firmware linkage (goes through [[re-firmware]])：
   • Protocol semantics unclear / custom payload → firmware extraction (binwalk unpacking) → find protocol implementation in rootfs (strings / so), keys, hardcoded topics → [[re-fw-emulate]] emulate device to reproduce protocol behavior
   • Cross-validate protocol constants and firmware strings (port, magic, topic prefix)
   • The OTA channel is also a protocol surface: MQTT topic ota/update delivers firmware → the intercepted firmware image can then go through [[re-firmware]]

BLE Link Layer

Complement the application-layer view with a link-layer perspective: over-the-air sniffing, advertising/connection frame structure, pairing encryption negotiation, and key localization. For the packet capture tool foundation, see the above "BLE —— bluez + ubertooth"; this section focuses on link semantics and deep-dive points.

Sniffing hardware and software (generalized purchasing guidance)

• The host side (btmon / Android btsnoop) can only see processes the local machine participates in; interactions between a device and third-party devices require over-the-air sniffing
• nRF-series dongle + Sniffer firmware + Wireshark extcap (mainly desktop): when purchasing, focus on the Bluetooth Core version covered by the chip (whether 4.2 / 5.x supports LE Secure Connections tracking), whether the firmware supports connection event tracking (following frequency hopping), and the antenna form (onboard/external, affecting reception distance)
• ubertooth-class 2.4GHz sniffers: suitable for link-layer data of advertising and unencrypted connections; they do not participate in connections, and encrypted connection contents cannot be seen
• Common limitation: for encrypted connections, over-the-air sniffing can only see timing and packet lengths; decrypting the content requires the LTK (see below)

Advertising packet parsing

• PDU types distinguish connection intent: ADV_IND (connectable undirected, most common), ADV_SCAN_IND (scannable), ADV_NONCONN_IND (pure advertising), ADV_DIRECT_IND (directed)
• Frame structure: 2-byte header (PDU type + length) + Payload (AdvA + AdvData); AdvData is an AD structure chain (Length + AD Type + Data): 0x01 Flags, 0x03/0x07 service UUID, 0x09 complete local name, 0x0A Tx Power, 0xFF manufacturer data
• Parsing: tshark -r ble.pcap -Y btle -T fields -e btle.advertising_address -e btle.advertising_header.pdu_type -e btle.advertising_data(field names are based on tshark -G fields | grep -i btle ); the 0xFF manufacturer data segment often contains proprietary protocols and device identifiers, and is worth byte-by-byte comparison with firmware
• Advertising only carries low-frequency status notifications (name/status/service discovery); business data is generally in the GATT channel after connection

Connection event timing and channels 37/38/39

• Primary advertising channels are fixed at 37/38/39 (2402/2426/2480 MHz), and advertising events rotate across the three channels; scan requests/responses and CONNECT_IND are also on these three channels
• CONNECT_IND carries the access address, hop increment, and connection parameters → from these, the subsequent data channel sequence (0-36) and event rhythm can be inferred
• Connection parameters: connInterval (1.25ms step), slaveLatency, supervisionTimeout, visible in the HCI-layer LE Connection Complete event; connection events start from an anchor point and occur periodically according to connInterval
• Verification: in packet captures, the interval between connection data packets should be an integer multiple of connInterval; chaotic intervals or one-sided absence → frequency hopping tracking is out of sync (see pitfall 3)

Pairing / encryption negotiation

• Flow: Pairing Request/Response (SMP) → temporary key derivation → LTK generation → (bonding) both parties store the LTK
• LE Legacy: derives STK based on PIN/temporary value; LE Secure Connections: P-256 ECDH, the MITM bit of AuthReq determines whether man-in-the-middle protection is in place (Just Works has no user verification, Passkey / Numeric Comparison do)
• Locating point: in Wireshark, look at AuthReq in the SMP exchange (SC / MITM / Bonding bits, fields according to tshark -G fields | grep -i smp ) — first confirm the mode, then decide whether to decrypt or only look at timing
• After bonding, reconnection does not re-pair; LTK exchange only appears during the first pairing
• Key storage: host side (system Bluetooth configuration / key-value storage) and device side (flash / external EEPROM) → [[re-firmware]] / [[re-crypto-keys]] extraction; once you have the LTK, fill it into Wireshark BLE preferences to decrypt connection data

GATT Service and Characteristic Enumeration

• Passive: btmon / btsnoop capture service discovery (Discover All Primary Services / characteristic discovery), and the btatt layer UUID-to-handle mapping is directly visible
• Active (after connecting): gatttool-like tools (bluez)primary / characteristics list handle ↔ UUID ↔ attributes
• Read/write/notify points: characteristic handle + CCCD (0x2902) after enabling notifications/indications, the value stream is in btatt.value;tshark -r ble.pcap -Y btatt -T fields -e btatt.handle -e btatt.value

Pitfalls and Traps

• Confusing advertising vs connection data: Symptom — seeing suspected business data in the manufacturer data section of an advertising packet and parsing it as reporting semantics, resulting in a mismatch with App behavior; Cause — advertising only carries low-frequency announcements, while the business flow is in the GATT channel after connection and may be encrypted; Countermeasure — first triage by frame type: parse AdvData for btle advertising packets, and look at the btatt layer for connection data; determine whether the target device has entered the connected state before deciding which segment to capture
• Misjudging pairing mode: Symptom — proceeding on the assumption that "capturing the pairing process is enough to decrypt", when in reality the connection is either fully ciphertext or plaintext with no verification; Cause — the AuthReq bits determine the mode: SC=0 and MITM=0 is LE Legacy Just Works — no user verification, no protection against eavesdropping-side impersonation; SC=1 is LE Secure Connections (ECDH); Countermeasure — parse the AuthReq of the SMP Pairing Request to first confirm the mode, then choose decryption (requires LTK) or only look at timing
• Packet capture missing connection events: Symptom — both advertising and connection establishment are present, but connection events are intermittent or only one side's packets are seen; Cause — data channel frequency hopping is not being tracked (missing the anchor point means losing synchronization), only a single channel is monitored in a fixed manner, or over-the-air interference causes packet loss; Countermeasure — prioritize host-side btmon / btsnoop (reliable across all channels); for over-the-air sniffing use firmware with connection tracking capability, and verify tracking against the CONNECT_IND access address + hop increment
• The LTK is not in the capture: Symptom — after the SMP exchange there are no LTK-related packets, so there is no key for decryption; Cause — after bonding, reconnection directly uses the stored key and does not exchange it again; Countermeasure — clear pairing/reset and re-capture the first pairing, or extract from device flash / App storage ([[re-firmware]] / [[re-crypto-keys]])

NFC / Smart Cards

Contact / contactless smart cards: ISO14443 link, APDU interaction, and common card family weaknesses. For hardware, use a general-purpose reader/writer such as proxmark3 (generalized purchasing); confirm the authorization boundary before capture/analysis.

ISO14443 Frame Structure and Anticollision Flow

• ISO14443A flow: REQA (0x26 short frame) → ATQA → anticollision (UID negotiated bit by bit through cascade levels CL1/CL2/CL3) → SELECT → SAK (card type identifier)
• Frame structure: SOF + data bytes (each byte with parity) + CRC_A (2 bytes) + EOF; short frame 7 bits (REQA/WUPA)
• SAK type determination: 0x08/0x88 class → MIFARE Classic family (Crypto-1, sector/block structure); 0x20/0x40 class → DESFire family (file system + AES/3DES) — determine type first, then choose the path (see Pitfall 1)
• Reader workflow: hf 14a reader class commands (proxmark3-class generic) output ATQA / UID / SAK in one go, then enter the corresponding card-family tool based on these

APDU interaction

• ISO7816-4 APDU: CLA + INS + P1 + P2, followed by Lc/data/Le appearing in case combinations (Case1 has none of the three; Case2 has no Lc/data but may have Le; Case3 has Lc+data and no Le; Case4 has all) — determine the case before parsing boundaries; response SW1/SW2 (0x9000 success)
• Common commands: 0xA4 SELECT (select application), 0xB0 READ BINARY, 0xB2 READ RECORD, 0x20 VERIFY (password verification), 0xD0 WRITE BINARY, 0xD6 UPDATE BINARY — but command sets differ across cards/applications (see Pitfall 2)
• Contact-based uses ISO7816 T=0/T=1; contactless cards often pass APDU through transparently (e.g., DESFire ISO mode), and the packet capture position differs
• Localization tip: capture known legitimate interactions (reader logs / phone NFC logs) and compare the command sequences, which is faster than guessing against documentation

MIFARE Classic and Crypto-1 weaknesses (generalized)

• Crypto-1 is a 48-bit stream cipher; during authentication the reader sends a challenge and the card returns an encrypted response; the PRNG and authentication protocol have known weaknesses, allowing key recovery based on non-cryptographic authentication response samples (replay / known-plaintext approach)
• Prerequisite: the card responds to authentication requests from any reader (not configured to reject unknown keys); if the prerequisite is not met, this approach does not apply — switch to key extraction ([[re-crypto-keys]] / [[re-firmware]]) or firmware analysis
• Boundary: only applies to Crypto-1 class cards; AES class cards (DESFire family) have a different structure and this does not apply

proxmark3-class device workflow (generalized purchasing)

• Purchasing considerations: 13.56MHz high-frequency support (essential), whether firmware is actively updated, whether on-site flashing is supported, antenna performance and enclosure form factor
• Workflow: power-on self-test → hf 14a reader type determination → choose attack/read-write workflow by card family → verify results
• Authorization boundary: both read-only and card writing/cloning require authorization; in a lab environment confirm card ownership and purpose ([[re-analyze/platform-tips]] highest principle)

Pitfalls and traps

• Incorrect card type determination (MIFARE vs DESFire have different structures): Symptom — parsing a card according to Classic sector/block structure, the addresses and data all fail to match; Cause — Classic (Crypto-1, sector-block structure, SAK 0x08/0x88 class) and DESFire (file system, AES/3DES, SAK 0x20/0x40 class) have completely different internal structures; Countermeasure — first look at SAK (combined with ATQA) to determine the card family, then choose the parsing/attack path; do not apply one structure to all cards
• APDU command set differences: Symptom — the same command succeeds on card A but card B returns errors like 0x6A82 (file not found) / 0x6D00 (instruction not supported); Cause — different cards/applications have different conventions for CLA prefix, INS, P1/P2 (proprietary CLA, extended instructions, differing parameter meanings); Countermeasure — first SELECT the target application, capture known legitimate interactions to compare command sequences, and calibrate item by item according to SW status codes
• Crypto-1 attack prerequisites: Symptom — the key recovery process does not run or produces wrong results; Cause — the approach requires first obtaining a non-encrypted authentication response sample from the card side, and there is no starting point when the card rejects authentication from an unknown reader or the environment cannot insert a man-in-the-middle; Countermeasure — first verify whether the card responds to authentication requests from any reader; if the precondition holds, proceed with the attack flow, otherwise switch to the key extraction / firmware path ([[re-crypto-keys]] / [[re-firmware]])
• UID and data block confusion: Symptom — after writing/copying the card, the target card behaves abnormally or read/write fails; Cause — the UID and the manufacturer block (block 0) have locking/verification, the UID of some card families cannot be changed, and after changing it the access control becomes invalid; Countermeasure — before writing, first read the manufacturer block and the sector trailer block (access bits) to confirm writability and authorization boundaries

Cross-domain combination

• [[re-protocol]]: this skill is its IoT branch (the choice-tree entry after step 5 of the workflow)
• [[re-netcap]]: capture raw material (capture points / isolation)
• [[re-firmware]]: firmware linkage (protocol implementation / keys / OTA images, together with [[re-fw-extract]] / [[re-fw-emulate]])
• [[re-crypto-id]] / [[re-crypto-keys]] / [[re-crypto-decrypt]]: encrypted traffic chain (TLS / DTLS / PSK key extraction and decryption)
• [[re-proto-rev]]: private IoT protocol state machine reconstruction
• [[re-hardware-io]]: serial port / flash chip key reading
• [[re-sandbox]]: device test network isolation and authorization boundaries (the highest principle of [[re-analyze/platform-tips]])
• This skill is referenced by the triage "protocol analysis" path of [[re-analyze]] (re-protocol → re-iot-proto)
• Radio-frequency signal-level reverse engineering (acquisition/demodulation/frame recovery) → [[re-sdr]]
• [[re-javacard]]: applet logic on top of the NFC/smart card chain (CAP file parsing, process(APDU) dispatch reconstruction)

Common pitfalls and traps

• MQTT plaintext topic leaks control semantics: Symptom — in the pcap the topic directly states device/x/ota/update、cmd/relay, the business intent is clear at a glance but the payload is encrypted or binary; Cause — devices often encrypt only the payload while the topic is plaintext (or the broker has no TLS); Countermeasure — the topic is a semantic goldmine: first -e mqtt.topic | sort | uniq -c bucket them, locate key flows by topic and then decrypt the payload; for binary payloads, compare JSON/protobuf assumptions against the firmware (step 5)
• DTLS/PSK encryption requires keys: Symptom — 5684 / 8883 traffic has high entropy and is all ciphertext, and the dissector outputs only garbage; Cause — DTLS/TLS encryption layer (PSK/RSA); Countermeasure — extract the PSK from the device firmware / App side ([[re-crypto-keys]] → [[re-firmware]]), enter the hexadecimal PSK in the Wireshark dtls preferences; for TLS use SSLKEYLOGFILE (the RSA private key is usable only for legacy static-RSA sessions, and modern ECDHE/TLS 1.3 cannot be decrypted); if the key cannot be obtained, this channel can only be examined for timing, so mark the limitation and switch to semantic inference
• BLE whitelist/pairing binding: Symptom — after the connection request there is no subsequent data, or the device does not broadcast at all; Cause — the whitelist filters unknown hosts, and already-paired devices connect directly according to bonding information (skipping the broadcast / pairing process); Countermeasure — clear the device pairing / factory reset and then re-capture the first connection pairing process; first fully capture the broadcast-phase data (adv packets, including device information and service UUIDs)
• Zigbee network key acquisition is difficult: Symptoms — zbee frames can be parsed but APS / ZCL are fully encrypted (payload is garbled); Cause — the network key is not in hand, link/network layer encryption; Countermeasures — try each item along the key path in step 4 one by one: packet capture during pairing/commissioning → default TC link key → firmware extraction (use zbgodfind to scan firmware to find the key); both layers of keys, the TC link key and the network key, must be obtained
• Packet capture hardware/channel mismatch: Symptoms — BLE only captures sporadic advertisements, and Zigbee captures not a single packet; Cause — BLE frequency hopping across 40 channels (ordinary adapters can only see advertisements or some connection packets), Zigbee channel selected incorrectly (choosing the wrong one among 11-26 means silence); Countermeasures — for BLE use btmon / btsnoop (host-side all channels) or ubertooth with follow; for Zigbee use a sniffer channel scan (zbstumbler) to confirm the working channel before capturing