rust-syntax-macros-procedural

rust-syntax-macros-procedural

The mechanics of writing procedural macros in Rust: how to scaffold a proc-macro crate, which of the three entry-point attributes (#[proc_macro], #[proc_macro_derive], #[proc_macro_attribute]) the compiler will accept, how to parse the input TokenStream with syn, how to build the output TokenStream with quote!, how to surface a compile error with syn::Error::new_spanned, and why every non-entry-point function in your codebase MUST use proc_macro2::TokenStream instead of the compiler-side proc_macro::TokenStream.

Cross-references: [[rust-syntax-macros-declarative]] (macro_rules!, when declarative beats procedural) [[rust-syntax-traits]] (derive macros almost always implement a trait) [[rust-impl-cargo-project]] ([lib] proc-macro = true) [[rust-syntax-edition-2024]] (#[diagnostic::do_not_recommend] stable in 1.85).

---

When to use this skill

• User writes #[proc_macro_derive(Foo)], #[proc_macro_attribute], or #[proc_macro] and needs the entry-point rules.
• User asks "do I need syn?" or "what is quote!?" or "why two TokenStream types?".
• User parses input via parse_macro_input! / syn::parse2 and needs to know which to pick.
• User wants to report a precise compile error pointing at one token (span-aware diagnostic).
• User wants to test proc-macro logic in a unit test (cannot use proc_macro::TokenStream outside a proc-macro crate).
• User wants to re-export a macro from a parent crate so end users do not depend on the implementation crate directly.

If the user only needs macro_rules! see [[rust-syntax-macros-declarative]]. If the user needs derive macros that implement a trait (Serialize, Error, etc.) read this skill first, then [[rust-syntax-traits]] for the trait being derived.

---

Quick reference: the three kinds

Kind	Attribute	Signature the compiler requires	Invoked as
Function-like	#[proc_macro]	pub fn name(input: TokenStream) -> TokenStream	name!(...)
Derive	#[proc_macro_derive(Name)]	pub fn fn_name(input: TokenStream) -> TokenStream	#[derive(Name)]
Derive with helpers	#[proc_macro_derive(Name, attributes(helper, ...))]	same as above	#[derive(Name)] + #[helper] on fields
Attribute	#[proc_macro_attribute]	pub fn name(attr: TokenStream, item: TokenStream) -> TokenStream	#[name(args)]

Source: Rust Reference: procedural-macros

Every entry point MUST be pub fn, at the crate root, with the compiler-supplied proc_macro::TokenStream. No async, no unsafe, no generics, only Rust ABI.

---

Quick reference: the four crates

Crate	Provided by	Where it can be used	Purpose
proc_macro	the compiler (no Cargo dep)	ONLY inside a crate with proc-macro = true	The entry-point TokenStream, returned to the compiler.
proc_macro2	crates.io	anywhere (lib, bin, build script, tests)	A drop-in mirror of proc_macro that is usable outside proc-macro crates.
syn	crates.io	anywhere (built on proc_macro2)	Parses a TokenStream into a typed Rust AST (DeriveInput, ItemFn, Type, ...).
quote	crates.io	anywhere (built on proc_macro2)	Builds a proc_macro2::TokenStream from quoted-Rust syntax with #var interpolation.

Source: docs.rs/proc-macro2 Source: docs.rs/syn Source: docs.rs/quote

ALWAYS write all body logic against proc_macro2::TokenStream. NEVER pass proc_macro::TokenStream around inside helper functions: it makes the helper untestable and unusable outside the proc-macro crate.

---

Mandatory crate setup

A procedural-macro crate is a separate crate. The Reference: "Procedural macros must be defined in a crate with the crate type of proc-macro."

# my_macros/Cargo.toml
[package]
name = "my_macros"
version = "0.1.0"
edition = "2024"

[lib]
proc-macro = true

[dependencies]
syn = { version = "2", features = ["full", "extra-traits"] }
quote = "1"
proc-macro2 = "1"

Source: Rust Reference: procedural-macros

NEVER put library code, types, or traits in this crate. The Reference: "Macros may not be used from the crate where they are defined." The standard topology is two crates:

my_thing/                  # public API crate (the one users add to [dependencies])
  Cargo.toml               # depends on my_thing_macros, re-exports its macros
  src/lib.rs               # pub use my_thing_macros::Foo;
my_thing_macros/           # proc-macro crate (proc-macro = true)
  Cargo.toml
  src/lib.rs               # entry points only

ALWAYS re-export macros from the public API crate. NEVER force end users to depend on the *_macros crate directly: every public symbol must come from one crate.

---

Function-like macro: minimal entry point

// src/lib.rs of the proc-macro crate
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, LitStr};

#[proc_macro]
pub fn shout(input: TokenStream) -> TokenStream {
    let s = parse_macro_input!(input as LitStr).value();
    let upper = s.to_uppercase();
    quote! { #upper }.into()
}

Usage in a consumer crate:

let banner = my_macros::shout!("hello");
assert_eq!(banner, "HELLO");

Source: Rust Reference: procedural-macros Source: docs.rs/quote

ALWAYS end the entry point with .into() to convert proc_macro2::TokenStream (what quote! returns) back into proc_macro::TokenStream (what the compiler accepts).

---

Derive macro: parse, build, return

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput};

#[proc_macro_derive(Hello)]
pub fn derive_hello(input: TokenStream) -> TokenStream {
    let DeriveInput { ident, generics, .. } = parse_macro_input!(input as DeriveInput);
    let (impl_g, ty_g, where_c) = generics.split_for_impl();

    let expanded = quote! {
        impl #impl_g ::core::fmt::Display for #ident #ty_g #where_c {
            fn fmt(&self, f: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
                ::core::write!(f, "hello from {}", ::core::stringify!(#ident))
            }
        }
    };
    expanded.into()
}

Source: Rust Reference: procedural-macros Source: docs.rs/syn

Notes:

• DeriveInput exposes ident, generics, data, attrs, vis. It is the universal struct-or-enum-or-union input to derives.
• generics.split_for_impl() returns the three pieces <...>, <...>, where ... you need to forward generics into the impl block. NEVER hand-spell the generics; you will drop bounds or get lifetime ordering wrong.
• Absolute paths (::core::fmt::Display, ::core::write!) defeat the unhygienic-by-default behaviour of proc macros (see Hygiene below).

Helper attributes

#[proc_macro_derive(Builder, attributes(builder))]
pub fn derive_builder(input: TokenStream) -> TokenStream { /* ... */ }

Usage:

#[derive(Builder)]
struct Req {
    #[builder(default)]    // helper attribute, declared above
    timeout_ms: u64,
}

Source: Rust Reference: procedural-macros

Helper attributes are inert. The compiler does not strip them off the input until your derive runs; they are visible to your macro and ignored otherwise. NEVER define a free-standing #[builder] attribute macro plus a #[derive(Builder)] derive macro that read the same attribute name: pick one mechanism.

---

Attribute macro: two TokenStreams

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, ItemFn};

#[proc_macro_attribute]
pub fn instrument(attr: TokenStream, item: TokenStream) -> TokenStream {
    let _args = attr; // parse as needed; empty when invoked as `#[instrument]`
    let input = parse_macro_input!(item as ItemFn);
    let sig = &input.sig;
    let block = &input.block;
    let name = &sig.ident;

    quote! {
        #sig {
            ::std::eprintln!(">> {}", ::core::stringify!(#name));
            let __out = (|| #block)();
            ::std::eprintln!("<< {}", ::core::stringify!(#name));
            __out
        }
    }.into()
}

The two inputs:

• attr is the tokens INSIDE the outer delimiters of #[instrument(foo, bar)], EXCLUDING the delimiters. For a bare #[instrument] it is empty.
• item is the entire item the attribute decorates (function, struct, impl block, etc.) PLUS any other attributes still attached to it.

Source: Rust Reference: procedural-macros

The macro returns ZERO OR MORE items that REPLACE the original. ALWAYS emit the original item (often unchanged) plus the wrapper; NEVER silently drop the input.

---

TokenStream conversion: the only place you mix worlds

// Entry point: compiler hands you proc_macro::TokenStream
#[proc_macro]
pub fn my_macro(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
    // Convert ONCE at the boundary.
    let input2: proc_macro2::TokenStream = input.into();

    // ALL helpers from here on use proc_macro2::TokenStream.
    let output2: proc_macro2::TokenStream = expand(input2);

    // Convert ONCE back to compiler type.
    output2.into()
}

fn expand(input: proc_macro2::TokenStream) -> proc_macro2::TokenStream {
    // testable in unit tests, no proc-macro crate context required
    quote::quote! { /* ... */ }
}

Source: docs.rs/proc-macro2

ALWAYS convert at the entry-point boundary, never inside helpers. This makes the expand function unit-testable from a regular #[test] because proc_macro2::TokenStream works anywhere.

---

Decision tree: how do I parse the input

Parsing input TokenStream:
|
+-- Whole derive input (struct / enum / union)?
|   -> parse_macro_input!(input as DeriveInput)
|
+-- Whole function item?
|   -> parse_macro_input!(input as ItemFn)
|
+-- Whole impl block / struct / module / trait?
|   -> parse_macro_input!(input as Item) and match the enum variant
|
+-- A single expression / type / literal / identifier?
|   -> parse_macro_input!(input as Expr | Type | LitStr | Ident)
|
+-- Custom DSL grammar you defined?
|   -> implement syn::parse::Parse, then parse_macro_input!(input as MyDsl)
|
+-- Comma-separated list of T?
|   -> Punctuated::<T, Comma>::parse_terminated

parse_macro_input! and syn::parse2 differ only in input type: the former takes proc_macro::TokenStream (only valid in the entry point), the latter takes proc_macro2::TokenStream (valid anywhere). USE parse2 in helpers.

Source: docs.rs/syn

---

Span-aware error reporting

A Span represents a region of source code. Diagnostics carry a Span so the compiler can underline the offending tokens.

use syn::{Error, spanned::Spanned};

fn require_named_struct(input: &syn::DeriveInput) -> Result<(), Error> {
    match &input.data {
        syn::Data::Struct(syn::DataStruct { fields: syn::Fields::Named(_), .. }) => Ok(()),
        other => Err(Error::new_spanned(
            input,
            "this derive only supports structs with named fields",
        )),
    }
}

Then in the entry point:

#[proc_macro_derive(MyDerive)]
pub fn derive(input: TokenStream) -> TokenStream {
    let ast = parse_macro_input!(input as DeriveInput);
    match expand(&ast) {
        Ok(ts) => ts.into(),
        Err(e) => e.to_compile_error().into(),
    }
}

Source: docs.rs/syn Source: Rust Reference: procedural-macros

ALWAYS use syn::Error::new_spanned(tokens, message) so the error points at the specific token range. NEVER use panic! inside a proc macro: the compiler still converts it into an error, but the error has no useful span and the panic message is wrapped in noise.

The two terminal calls are e.to_compile_error() (returns a TokenStream containing a compile_error! invocation) and the older e.into_compile_error() (consumes self). Either is correct; pick one and stay consistent.

---

Hygiene: proc macros are unhygienic

Declarative macro_rules! macros have mixed-site hygiene (local variables resolve at definition site, other symbols at call site). Procedural macros do not. The Reference: "Procedural macros are unhygienic. This means they behave as if the output token stream was written inline to the code it is next to."

Consequences and rules:

• ALWAYS reference standard items by absolute path: ::core::option::Option, ::std::vec::Vec, ::core::fmt::Formatter. The user might have shadowed Option in scope.
• ALWAYS prefer ::core::* over ::std::* so the macro works in #![no_std] crates.
• ALWAYS prefix locally-generated identifiers to avoid clashes with user code: __internal_field_count, not count.
• NEVER assume any item or trait is in scope at the call site; resolve every path explicitly.

For a span-changing trick that simulates "hygienic" identifiers, use quote_spanned!(span=> ...):

use quote::quote_spanned;
let span = field.span();
quote_spanned! { span =>
    _: ::core::clone::Clone + ::core::marker::Send,
}

Source: Rust Reference: procedural-macros Source: docs.rs/quote

---

Shaping errors with #[diagnostic::do_not_recommend] (1.85)

Edition 2024 stabilises #[diagnostic::do_not_recommend]. Apply it to a trait impl you do NOT want the compiler to suggest in error messages. Useful when a derive macro emits an auxiliary trait impl that should never appear in "trait X not implemented" advice.

// inside the derive's emitted code
quote! {
    #[diagnostic::do_not_recommend]
    impl #impl_g MyInternalHelper for #ident #ty_g #where_c { /* ... */ }
}

Source: Rust Reference: procedural-macros Source: Rust 1.85 release notes

ALWAYS hide internal "plumbing" impls behind this diagnostic when they are not part of the macro's user-facing contract.

---

Re-exporting macros from a parent crate

my_thing (public API crate):

[dependencies]
my_thing_macros = { path = "../my_thing_macros", version = "0.1" }

// my_thing/src/lib.rs
pub trait Hello { fn hello(&self) -> String; }
pub use my_thing_macros::Hello;   // re-export the derive

Now end users write:

[dependencies]
my_thing = "0.1"

and import use my_thing::Hello; ONCE; this single import brings BOTH the trait and the derive into scope.

ALWAYS make the trait and the derive share a name (Hello trait and Hello derive). NEVER let the proc-macro crate appear in your users' Cargo.toml: that splits the version contract.

---

Anti-patterns (full list in references/anti-patterns.md)

• Mixing proc-macro entry points and regular library types in one crate. The Reference forbids USING the macro from its defining crate; the convention is also to KEEP the proc-macro crate pure.
• Passing proc_macro::TokenStream through helper functions. Makes everything untestable.
• Forgetting [lib] proc-macro = true in Cargo.toml. The compiler then rejects every #[proc_macro*] attribute.
• Using panic! for user errors. Use syn::Error::new_spanned.
• Generating identifiers without considering hygiene (count, result, Option). Use absolute paths and __internal_* prefixes.
• Defining the same name as both a free attribute macro and a derive helper. Pick one mechanism.

---

Reference links

• Rust Reference: procedural-macros
• docs.rs: syn
• docs.rs: quote
• docs.rs: proc-macro2
• Rust 1.85.0 release notes
• Cargo manifest reference

For deeper drill-downs see:

• references/methods.md: every attribute / function / type signature verbatim from the docs.
• references/examples.md: complete working derive, attribute, and function-like macros end-to-end.
• references/anti-patterns.md: every common proc-macro mistake with root-cause explanations and fixes.