SuperAPI AI Directory

Lisp Macro Architecture

General & other

View repository

Updated 10-16 07:13

About this skill

<absolute_laws priority="critical"> Resolve everything resolvable at macro-expansion time: clause structure, state/register dependency graphs, lifetimes, continuation chains. Never reach for eval. Every fact known from the S-expression shape alone is a fact the runtime should never have to recompute. A DSL that defers this to eval pays a performance and safety tax on every execution instead of once at compile time. <how_to_apply>In the analyzer stage, build a static graph (plist/struct/alist) over the AST and answer questions ("is this register still live here?", "does this clause reference an undefined state?") by walking that graph, not by generating code that asks the question at runtime.</how_to_apply>

  (defmacro state-machine (name &amp;body clauses)
    "Define a compile-time-verified finite state machine NAME."
    (sm--emit (sm--analyze (sm--parse clauses))))
</example>
  ;; Emacs Lisp (requires -*- lexical-binding: t; -*-)
  (eval-and-compile
    (defun dsl--parse-clause (clause) ...))
</example>
  ;; Emacs Lisp (manual once-only, since cl-lib has no equivalent)
  (defmacro my-max2 (a b)
    (let ((ga (make-symbol "a")) (gb (make-symbol "b")))
      `(let ((,ga ,a) (,gb ,b))
         (if (&gt; ,ga ,gb) ,ga ,gb))))
</example>
  ;; Emacs Lisp
  (defun dsl--parse-clause (clause)
    (unless (and (consp clause) (symbolp (car clause)))
      (error "state-machine: expected (STATE-NAME . TRANSITIONS), got %S" clause))
    ...)
</example>
  ;; Emacs Lisp: declare indent + debug spec explicitly
  (defmacro with-resource (var resource-form &amp;rest body)
    "Bind VAR to RESOURCE-FORM for the dynamic extent of BODY."
    (declare (indent 1) (debug ((symbolp form) body)))
    (let ((gvar (make-symbol "resource")))
      `(let ((,gvar ,resource-form))
         (let ((,var ,gvar))
           (unwind-protect (progn ,@body)
             (close-resource ,gvar))))))
</example>
  ;; Common Lisp: intentional anaphoric capture, documented
  (defmacro aif (test then &amp;optional else)
    "Anaphoric IF. Binds IT to the value of TEST, visible inside THEN/ELSE.
     This capture is intentional; see anaphoric macro convention."
    `(let ((it ,test))
       (if it ,then ,else)))

  ;; Emacs Lisp: hygienic internal temp
  (defmacro my-swap (a b)
    (let ((tmp (cl-gensym "tmp-")))
      `(let ((,tmp ,a))
         (setf ,a ,b ,b ,tmp))))
</example>

<canonical_technique_library> The core repertoire for writing individually "correct" macros, drawn from Paul Graham's On Lisp and Doug Hoyte's Let Over Lambda. Apply these techniques to any single macro, independent of whether it is part of a larger DSL pipeline (see workflow/patterns above). Code below is Common Lisp; Elisp equivalents are noted where they diverge.

  ;; usage
  (defmacro my-max2 (a b)
    (once-only (a b)
      `(if (&gt; ,a ,b) ,a ,b)))
</example>
<note>The triple-backquote body is exactly the kind of nesting the ban_on_macro_monoliths law warns against — it is a deliberately-hardened, well-tested exception. Prefer a battle-tested library implementation (alexandria:once-only) over hand-rolling this in application code.</note>
  (defmacro awhen (test &amp;body body)
    `(aif ,test (progn ,@body)))

  (defmacro aand (&amp;rest args)
    (cond ((null args) t)
          ((null (cdr args)) (car args))
          (t `(aif ,(car args) (aand ,@(cdr args))))))

  (defmacro alambda (parms &amp;body body)
    "Anonymous function that can recurse via SELF."
    `(labels ((self ,parms ,@body))
       #'self))

  ;; usage: (alambda (n) (if (&lt;= n 1) 1 (* n (self (1- n)))))
</example>
  (defun g!-symbol-p (s)
    (and (symbolp s) (&gt; (length (symbol-name s)) 2)
         (string= (symbol-name s) "G!" :end1 2)))

  (defmacro defmacro/g! (name args &amp;rest body)
    (let ((syms (remove-duplicates
                 (remove-if-not #'g!-symbol-p (flatten body)))))
      `(defmacro ,name ,args
         (let ,(mapcar (lambda (s) `(,s (gensym ,(subseq (symbol-name s) 2)))) syms)
           ,@body))))

  (defun o!-symbol-p (s)
    (and (symbolp s) (&gt; (length (symbol-name s)) 2)
         (string= (symbol-name s) "O!" :end1 2)))
  (defun o!-symbol-to-g!-symbol (s)
    (symb "G!" (subseq (symbol-name s) 2)))

  (defmacro defmacro! (name args &amp;rest body)
    (let* ((os (remove-if-not #'o!-symbol-p args))
           (gs (mapcar #'o!-symbol-to-g!-symbol os)))
      `(defmacro/g! ,name ,args
         `(let ,(mapcar #'list (list ,@gs) (list ,@os))
            ,(progn ,@body)))))

  ;; usage: o! args are evaluated exactly once; g! symbols are auto-gensym'd
  (defmacro! my-max2 (o!a o!b)
    `(if (&gt; ,g!a ,g!b) ,g!a ,g!b))
</example>
<note>This is the idiomatic replacement for hand-writing once_only + gensym boilerplate on every macro. Prefer it once available in a project; the manual once_only_evaluation_order pattern above shows what it replaces.</note>
  (defmacro toggle (place)
    `(_f not ,place))

  ;; usage: (toggle (gethash 'k table)) or (toggle (aref v i)) — works on any place
</example>
<note>Standard CL also provides define-modify-macro for the common case of a fixed operator, e.g. (define-modify-macro appendf (&amp;rest args) append).</note>
  (defmacro =lambda (parms &amp;body body)
    `#'(lambda (*cont* ,@parms) ,@body))

  (defmacro =defun (name parms &amp;body body)
    (let ((f (intern (concatenate 'string "=" (symbol-name name)))))
      `(progn
         (defmacro ,name ,parms
           `(,',f *cont* ,,@parms))
         (defun ,f (*cont* ,@parms) ,@body))))

  (defmacro =bind (parms expr &amp;body body)
    `(let ((*cont* #'(lambda ,parms ,@body))) ,expr))

  (defmacro =values (&amp;rest retvals)
    `(funcall *cont* ,@retvals))

  ;; usage
  (=defun add1 (n) (=values (1+ n)))
  (=bind (result) (add1 41) (print result)) ; => prints 42
</example>
<note>This is the direct, textbook mechanism behind the "CPS transform" the Absolute Laws refer to: the continuation chain is built entirely by macro-defined plumbing (*cont*), and =defun/=bind read like ordinary function calls at every call site.</note>
  (defmacro tau (radius)                ; (TAU r) expands to (* 2 pi r)
    `(* 2 pi ,radius))

  tau        ; => 6.283185307179586
  (tau 5)    ; => 31.41592653589793
</example>
<note>Minimal illustration in the spirit of Let Over Lambda's duality-of-syntax technique, not a verbatim book example. Use sparingly — it trades a small amount of surprise for call-site brevity, and should be documented at the definition site.</note>

<fasl_safe_expansion> Constraints on what a macro may place into the code it returns, so the expansion survives file-compilation (compile-file / FASL dump) and self-hosted or descriptor-backed evaluation, not just interactive macroexpansion at the REPL. Interactive expansion hides these bugs because nothing is serialized.

  ;; good: let a function-designator SYMBOL flow through; funcall accepts it.
  (defmacro deftransform (name &optional (fn ''identity))
    `(defun ,name (x) (funcall ,fn x)))   ; ,fn splices the symbol IDENTITY
</example>

<binding_form_scope_reference> A macro that mechanically rewrites code — rename a symbol, extract a function, inline a binding, hoist a let — is only correct if it respects the exact scope, shadowing, and namespace rules of every binding form it walks. The most common macro/refactor bug is treating all parenthesized (name value) shapes as one uniform "binding": the safety of a mechanical edit depends on which namespace a name lives in and which sub-forms it shadows. This catalogs the rules that make such rewrites safe.

<namespace_rule name="value_vs_callable_bindings"> Distinguish value-place bindings from callable bindings, because a name in one namespace must not shadow a rename target in the other. <value_bindings>let, let*, symbol-macrolet, do/do*, prog/prog*, dolist, dotimes, with-slots, with-accessors bind value/place names. symbol-macrolet in particular is a value-place form: its names must not shadow function-call heads, and its expansion forms resolve in the outer environment while its body is shadowed by the symbol-macro names.</value_bindings> <callable_bindings>flet, labels, macrolet, compiler-macrolet introduce local callable bindings that DO shadow an outer callable rename target within their body scope.</callable_bindings> </namespace_rule>

<scope_rule name="init_vs_body_evaluation_environment"> For each binding form, know which sub-forms see the outer environment and which see the bound names — this determines which occurrences a rename may rewrite. let / prog: init forms are evaluated in the outer scope (parallel); the body sees all bound names. let* / do* / prog*: sequential — each init sees earlier bound names but not its own; the body sees all. do: init forms are outer-scope; step forms, the end-test, and the body see all iteration variables. dolist / dotimes: the list/count source is outer-scope; the iteration variable shadows the target in the optional result form and body. symbol-macrolet: expansion forms resolve in the outer environment; only body references count as in-scope references to the symbol macro (relevant to unused-binding and inline analyses). In quasiquote, preserve comma/comma-at prefixes and rename only the unquoted symbol-macro references. with-slots / with-accessors: the instance expression is outer-scope; slot/accessor names in specs are not value references; body references are the shadowed ones. Renaming a bare with-slots spec must expand it from slot to (new-name slot) to preserve the slot-name mapping. handler-bind / restart-bind: the spec head (condition type / restart name) and restart-bind option keywords (:report, :test, :interactive) are designators, not value references. A lambda in a handler function or restart-option position introduces parameters that shadow only that lambda's own body. </scope_rule>

<boundary_rule name="reader_prefix_and_quasiquote"> Treat quote and quasiquote reader prefixes as reference-literal boundaries, not merely (quote ...)/(list ...) heads. Under quote, symbols are data and must not be rewritten. Under quasiquote, data is protected but unquote (, / ,@) re-enters evaluation, so a rename must thread quasiquote depth: rewrite only unquoted references and preserve the , / ,@ prefixes on the replacement. #'symbol and (function symbol) are callable designators: rename them with the target in executable context, but skip them inside quasiquoted data unless an unquote re-enters evaluation. </boundary_rule>

<special_rule name="define_modify_macro_implicit_place"> define-modify-macro has an implicit leading place argument that precedes the user lambda-list parameters. Any call-site rewrite (add/remove/move/reorder an argument) must offset user arguments by one so the place argument is preserved. </special_rule> </binding_form_scope_reference>

<lambda_list_and_quasiquote_traps> Implementation-detail traps encountered when a macro system parses and re-emits macro lambda lists, and when tests construct expected expansions by hand.

<dialect_notes> Phase separation: eval-when (:compile-toplevel :load-toplevel :execute) Hygiene primitive: gensym (or alexandria:once-only / alexandria:with-gensyms for batches) Editor indentation: derived automatically from &body in the lambda list by SLY/SLIME; no explicit declare needed Verification: macroexpand-1 (single step) vs macroexpand (fully expand); use sly-macroexpand-1 / sly-macroexpand-all interactively Debug tooling: sly-macrostep (macrostep.el via SLY) for interactive step-through expansion Phase separation: eval-and-compile; requires -- lexical-binding: t; -- at file top Hygiene primitive: cl-gensym (from cl-lib) or make-symbol; gensym alone has no namespacing convention — always pass a descriptive prefix Editor indentation/debugging: explicit (declare (indent N) (debug SPEC)) as the first form in the macro body — this is not automatic Verification: macroexpand-1, macroexpand-all (from cl-macs); interactively via pp-macroexpand-last-sexp or macrostep.el Byte-compilation: compile-time helpers not wrapped in eval-and-compile fail silently to a runtime call instead of erroring — always verify with emacs --batch -f batch-byte-compile </dialect_notes>

<decision_tree name="macro_vs_function"> Does this abstraction need to control evaluation (timing/order/whether an argument is evaluated at all), capture syntax, or generate code shaped by its arguments? <if_yes>Use a macro — but keep it a thin pipeline hook over pure parse/analyze/emit functions</if_yes> <if_no>Use an ordinary function or higher-order function; a macro here only adds hygiene risk and debugging friction for no semantic benefit</if_no> </decision_tree>

<best_practices> Reach for the canonical_technique_library first (once-only, anaphora, defmacro!/g!-o!-symbols, generalized variables, CPS macros) instead of inventing ad hoc gensym/evaluation-order handling Design the input S-expression and hand-write its ideal expansion before writing any macro code (Phase 1 proof-first) Keep defmacro bodies to a 1-3 line pipeline call; all logic lives in testable functions Gensym every internal temporary; document every intentional anaphoric capture Validate DSL clause shape in the parser and signal errors before the emitter runs Bind every user-supplied argument form exactly once, in left-to-right order, before referencing it in the expansion Wrap compile-time helpers in eval-when (CL) or eval-and-compile (Elisp) Use &body in CL lambda lists for trailing body arguments; use (declare (indent ...) (debug ...)) in Elisp macros Verify every non-trivial macro's expansion with macroexpand-1/macroexpand against the Phase-1 proof before considering it complete Push static analysis (liveness, dependency graphs, non-determinism enumeration) into the analyzer stage, never into emitted runtime code Splice function designators (symbols), never #'fn function objects, into macro output so expansions remain FASL-dumpable Before mechanically renaming/extracting across a binding form, confirm its namespace (value vs callable) and which sub-forms it shadows Thread quasiquote depth in code-walking rewrites; preserve , / ,@ prefixes and leave quoted data untouched Reimplement standard binding macros (e.g. with-standard-io-syntax) against the full ANSI variable set, not a convenient subset </best_practices>

<anti_patterns> Three or more nested backquote/comma levels directly inside defmacro Extract a pure emit function that builds and returns the S-expression; keep defmacro to a pipeline call

<error_escalation inherits="core-patterns#error_escalation"> Missing &body/indent metadata causing cosmetic mis-indentation Multiple evaluation of a side-effecting argument form Variable capture bug from a missing gensym, silently breaking a caller Compile-time helper not phase-separated, causing macroexpansion to fail only in fresh/cross-compiled environments </error_escalation>

<related_skills> CLOS, ASDF, condition-system, and general Common Lisp fundamentals underlying macro design Emacs Lisp fundamentals, use-package, and editor integration this skill's DX laws build on macroexpand/trace/inspect workflows for verifying and debugging expansions at runtime Evidence-driven debugging methodology for tracking down capture/evaluation-order bugs Symbol navigation across macro definitions and their call sites </related_skills> <related_agents> Locate existing macro definitions and call sites in this skill domain Review macro hygiene, evaluation-order, and diagnostics quality against this skill's laws Flag backquote-monolith macros and suggest parser/analyzer/emitter decomposition </related_agents>