SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsmodel-schema-validation

model-schema-validation

Testing & quality

View repository

Updated 10-26 09:19

About this skill

Skill: Model Schema Validation Guide

Use this guide to define and enforce structural constraints (shapes, data types, runtime constraints) on loaded ExecuTorch .pte models using validateSpec.

🔍 Why Validate Model Specs?

Every ExecuTorch model exposes a schema (ModelSpec<ConcreteDim>), derived either from standard ExecuTorch MethodMeta at load time or from an exported get_model_schema companion method returning a JSON model spec.

To prevent runtime crashes, type mismatches, and memory allocation errors in C++, TypeScript task pipelines validate the loaded model's exported schema against allowed spec variants using validateSpec before allocating static tensors or executing inference.

🛠️ Validation API Reference

import {
  validateSpec,
  method,
  f32,
  i64,
  i32,
  DynamicDim as Dyn,
  constraint,
} from '../../../core/schema';

const { variant, dims } = validateSpec(model.schema, {
  batched: method(
    'forward',
    [i64(1, Dyn('L')), i64(1, Dyn('L'))],
    [f32(1, 'D')],
    [
      constraint.equality(
        { paramSide: 'input', tensorIdx: 0, dimIdx: 1 },
        { paramSide: 'input', tensorIdx: 1, dimIdx: 1 }
      ),
    ]
  ),
  unbatched: method(
    'forward',
    [i64(Dyn('L')), i64(Dyn('L'))],
    [f32('D')],
    [
      constraint.equality(
        { paramSide: 'input', tensorIdx: 0, dimIdx: 0 },
        { paramSide: 'input', tensorIdx: 1, dimIdx: 0 }
      ),
    ]
  ),
});

const [D] = dims.constant('D');
const L = dims.range('L');

Key Schema Utilities from src/core/schema.ts:

  • method(name, inputs, outputs, runtimeConstraints?): Constructs a method specification.
  • f32(...) / i64(...) / i32(...) / ui8(...): Shorthand helpers for tensor parameter specs.
  • StaticDim('symbol') / String Literals: Strings passed to shape helpers (e.g. 'H', 'W') automatically map to StaticDim, acting as static dimension wildcards. They bind strictly to constant positive integer dimensions in the exported spec.
  • DynamicDim('symbol') (or Dyn('symbol')): Creates a dynamic dimension symbol. Must be used when a dimension genuinely varies at runtime and binds to a range or enum domain in the exported spec.
  • Constraint Helpers (constraint):
    • DimRef Object Literal ({ paramSide: 'input' | 'output', tensorIdx, dimIdx }): Explicit reference to a tensor's dimension.
    • constraint.equality(...dims): Creates an equality constraint requiring the referenced dimensions to take the exact same value at runtime.
    • constraint.linear(lhs, rhs, a, b?): Creates a linear constraint lhs = a * rhs + b.
  • validateSpec(exportedSchema, allowedVariants): Compares the model's exported schema against named variants and returns { variant, dim, dims }.
  • Symbol Accessors (dims & dim):
    • dims.constant('N', 'H'): Extracts constant values for symbols as numbers.
    • dims.range('S'): Extracts range domains { min, max, step }.
    • dims.enum('E'): Extracts enum choices readonly number[].
    • dims.dynamic('L'): Extracts dynamic range or enum as raw ConcreteDim.
    • dims.any('D'): Extracts raw dimension value (number, Range, readonly number[], or ConcreteDim).
    • dim('N'): Extracts value for a single symbol.

📏 Symbolic Dimensions & Dynamic Shapes

Tensors can have static dimensions (integers), static symbol wildcards (strings), or dynamic symbols (DynamicDim). The validateSpec utility matches allowed variants in order:

  1. Numbers (Static Match): Must match the exact static integer in the exported spec.

  2. Strings / StaticDim (Static Wildcard Match): Binds string symbol names (e.g. 'H') to static constant integers in the exported spec. Repeated occurrences of the same static symbol must bind to the same constant value.

  3. DynamicDim (Dynamic Domain Match): Binds dynamic symbol names to range or enum domains in the exported spec.

  4. Variant Selection: Specify multiple variant keys in validateSpec (e.g. batched vs unbatched). The validator tests variants sequentially and returns the first matching key and bound symbols.

🔀 Dimension Domains vs. Runtime Constraints

Understanding the distinction between a dimension's domain and its runtime value is central to schema design and validation:

1. Dimension Domains & Domain Matching

  • Dimension Domain: The set of values a dimension may take:
    • constant: Exactly value (a singleton integer).
    • range: Any value between min and max stepping by step.
    • enum: Any value listed in choices.
  • Runtime Value: The concrete integer size of a tensor's dimension in a specific execution call.
  • Domain Matching (validateSpec):
    • validateSpec performs static, load-time domain matching.
    • Dynamic symbols (DynamicDim('S')) bind to exported dimension domains. Reusing a symbol ('S') across tensor inputs or outputs requires every occurrence to bind to the exact same domain.
    • ⚠️ Key Rule: Binding to the same domain does NOT mean runtime values coincide! Two dimensions bound to the same domain (e.g., both having range 1..512) may take different runtime values in a single execution (e.g. length 10 and length 25).

2. Runtime Constraints (constraint.equality & constraint.linear)

  • Runtime Constraints: Declarations about the runtime values of tensor dimensions during execution:
    • Equality Constraint (constraint.equality(...)): Requires all referenced dimensions to take the exact same runtime value in any execution call. 
      method(
  'forward',
  [f32('B', Dyn('S1')), f32('B', Dyn('S2'))],
  [f32('B', Dyn('S1'))],
  [
    constraint.equality(
      { paramSide: 'input', tensorIdx: 0, dimIdx: 0 },
      { paramSide: 'input', tensorIdx: 1, dimIdx: 0 }
    ),
  ]
);
    • Linear Constraint (constraint.linear(...)): Requires two dimensions to satisfy dimLhs = a * dimRhs + b at runtime.
  • Validation & Enforcement:
    • validateSpec verifies that the exported model spec declares the exact same runtime constraints (1-to-1 declaration match).
    • Native C++ validates input runtime constraints before invoking model.execute().

⚙️ Companion JSON Spec (get_model_schema)

For models with runtime dynamic dimensions or runtime constraints (equality/linear relations between dimensions), the .pte model exports a companion method named get_model_schema returning a JSON string representation of ModelSpec<ConcreteDim>.

When get_model_schema is present, the C++ loader parses it to populate model.schema with exact RangeDim / EnumDim domains and runtime constraints. Without it, model.schema is populated strictly from static ExecuTorch MethodMeta.

📋 Common Validation Recipes

1. Classification (Batched vs Unbatched)

Accepts an image tensor and outputs class probabilities:

const { variant, dims } = validateSpec(model.schema, {
  batched: method('forward', [f32(1, 3, 'H', 'W')], [f32(1, 'N')]),
  unbatched: method('forward', [f32(3, 'H', 'W')], [f32('N')]),
});

const [N, H, W] = dims.constant('N', 'H', 'W');
const inpShape = { batched: [1, 3, H, W], unbatched: [3, H, W] }[variant];
const outShape = { batched: [1, N], unbatched: [N] }[variant];

2. Image-to-Image / Style Transfer

Accepts an image tensor and returns a modified image tensor with identical dimensions:

const { dims } = validateSpec(model.schema, {
  default: method('forward', [f32(1, 3, 'H', 'W')], [f32(1, 3, 'H', 'W')]),
});

const [H, W] = dims.constant('H', 'W');

3. Object Detection (Dynamic Box Count)

Accepts an image tensor, and outputs boxes, scores, and class labels for N detections:

const { dims } = validateSpec(model.schema, {
  default: method('forward', [f32(1, 3, 'H', 'W')], [f32('N', 4), f32('N'), f32('N')]),
});

const [N, H, W] = dims.constant('N', 'H', 'W');

4. Single-Model Pipeline (Fully Static Export)

Assert exact shape constants from the task file:

const { dims } = validateSpec(model.schema, {
  default: method(
    'denoise',
    [
      f32(1, LATENT_CHANNELS, LATENT_SIZE, LATENT_SIZE),
      i64(1),
      f32(1, CLIP_MAX_TOKENS, CLIP_HIDDEN_SIZE),
    ],
    [f32(1, LATENT_CHANNELS, LATENT_SIZE, LATENT_SIZE)]
  ),
});

🚫 Avoid / Anti-Patterns

  • Do NOT write manual imperative shape/type checks: Always use declarative validateSpec variants which report unified, readable mismatch errors.
  • Do NOT use hardcoded integers for dynamic dimensions: Use string symbols (like 'H', 'W', 'N') for dynamic dimensions. Conversely, fixed export dimensions should be integers.
  • Do NOT skip validation at startup: Validate model.schema before allocating static tensors.
  • Do NOT skip validation for single-model pipelines: Single-model pipelines should still validate against task shape constants.

⚠️ Failure Codes

validateSpec and the native spec validation raise SCHEMA_MISMATCH (the spec-builder helpers ConstantDim / RangeDim / EnumDim raise INVALID_ARGUMENT for their own arguments). A runtime constraint violated by the tensors actually passed to execute raises INVALID_ARGUMENT, not SCHEMA_MISMATCH: the model is fine, the call is not. See the Error Handling Skill.

📋 Verification Checklist

When specifying model schema validations, verify that:

  • Schema validation is performed immediately after model loading and before tensor initialization using validateSpec(model.schema, { ... }).
  • Dynamic dimensions are defined as string symbols, while fixed dimensions are plain integers.
  • Symbol values are extracted using dims.constant(...), dims.range(...), or dims.enum(...).
  • Multiple shape variants (e.g. batched vs unbatched) are provided when supported.
  • Input and output constraints map accurately to model specifications.
  • New validation failures throw SCHEMA_MISMATCH (or INVALID_ARGUMENT when the caller's own arguments are at fault), never a bare Error.