SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsflaggems-pr-submit-flagos

flaggems-pr-submit-flagos

Coding & development

View repository

Updated 10-15 21:42

About this skill

FlagGems Operator PR Submission Skill

Submission workflow: canonical name lookup → create branch → extract worktree code (6 files) → script validation → pre-commit → push → create PR → backfill link.

Rules (violations will cause the PR to be rejected)

25+ checks are already check_operator.py automatically executed (see the checklist below); the following only listsrules the model needs to actively pay attention to.

Workflow Rules

  1. Run the script before commit — check_operator.py --strict must have 0 errors
  2. Use canonical naming — before submitting, use operator_registry.py lookup to look up
  3. Backfill the PR link — after the PR is created, must operator_registry.py backfill
  4. PR description is generated by the script — gen_pr_description.py output JSON, mapped to the template (English)

Code Rules

  1. Code must be consistent with the original worktree version — rewriting test logic is not allowed; only import adjustments and formatting are allowed
  2. Do not delete existing comments in the worktree
  3. Underscore naming — for operators with a leading _ , remove the underscore from the mark/yaml id/file name, keep the rest (see references/naming.md）
  4. dtype defaults to constants — tests use utils.FLOAT_DTYPES, benchmarks use consts.FLOAT_DTYPES; if CUDA does not support it, it may be hardcoded but a comment must be added
  5. Non-pointwise benchmark — use GenericBenchmark(input_fn=...)for simple cases; when a custom shape is needed, inherit and override set_shapes
  6. Hardcoded size requires a comment — kernel BLOCK, test shapes, and benchmark shapes all require comments explaining the reason
  7. Add an assert in the wrapper for unsupported dtypes
  8. Split overloaded ops yaml into separate entries — refer to the eq / eq_scalar pattern
  9. Prohibit .is_cuda — use flag_gems.devicefor device checks, do not use .is_cuda or device.type == "cuda"(multi-backend)
  10. Put Autotune configs in the config file — do not inline hardcode autotune configs in the kernel
  11. Logger format — logger.debug("GEMS <OP_NAME_UPPER>"), do not use other formats
  12. Put Fused operators in the fused/ directory — src/flag_gems/fused/, do not put them in src/flag_gems/ops/

Submission Rules

  1. Do not modify existing upstream tests — only add, do not change existing functions
  2. Submit the general version first, then the specialized version
  3. Use statistical validation for probabilistic operators — mean ≈ p, do not only check 0/1
  4. Use gems_assert_close(equal_nan=True) for nan comparison

Environment

Environment Variable Required Description Default Value
FLAGGEMS_REPO Yes Local path to the FlagGems repository —
GH_TOKEN Yes GitHub Personal Access Token —
FLAGGEMS_FORK No Fork repository (automatically inferred from git remote) XDYuanzhuLee/FlagGems
FLAGGEMS_UPSTREAM No Upstream repository flagos-ai/FlagGems
FLAGGEMS_NORM_XLSX No Canonical name Excel path /workspace/规范名.xlsx
FLAGGEMS_PR_XLSX No Excel path of the operator to be submitted /workspace/第一批pr算子.xlsx
FLAGGEMS_DOMESTIC_DIR No Domestic GPU test data directory /workspace/国产GPU算子测试情况

Each new session must ensure that FLAGGEMS_REPO and GH_TOKEN are set (hardcoding is prohibited).

Workflow (the model only needs to call 3 commands)

Phase 0: Name Lookup

cd $FLAGGEMS_REPO
python <SCRIPTS_DIR>/operator_registry.py lookup <op>

Phase 1: Preparation

cd $FLAGGEMS_REPO
git checkout -b pr/<op> upstream/master

Confirm the operator does not exist upstream. Never cherry-pick or rebase.

Phase 2: Extract Code (completed in one step, manual writing prohibited)

python <SCRIPTS_DIR>/extract_from_worktree.py <op> --repo-dir $FLAGGEMS_REPO

The script automatically extracts 6 files from the worktree: kernel, test, benchmark, ops/init.py、init.py、operators.yaml。 All registrations are inserted in alphabetical order, and all code is extracted from the worktree as-is. Manually writing test/benchmark code is prohibited.

After the script completes, check whether the description in operators.yaml needs to be supplemented (the script may use a default placeholder description).

Phase 3-7: Validate, Test, Submit (completed in one step, manual skipping prohibited)

CUDA_VISIBLE_DEVICES=<N> python <SCRIPTS_DIR>/submit_operator.py <op> --repo-dir $FLAGGEMS_REPO

The script serially executes 9 steps (including 1.5 multiple-overload check): check_operator → multiple-overload consistency check → pre-commit → Local test → Local benchmark → PR description generation → commit → push → create PR → backfill link. If any step fails, interrupt and exit immediately. Manually executing individual steps to bypass this is not allowed.

Optional parameters:

  • --dry-run — validate only, do not submit (for debugging)
  • --skip-test — skip local tests (only when the environment does not support them)
  • --skip-benchmark — skip benchmark, only check domestic card data

References

  • references/workflow.md — detailed six-file template for Phase 2, code review points
  • references/pr-template.md — PR description template, JSON field mapping
  • references/naming.md — underscore operator naming rules comparison table
  • references/pr-checklist.md — pre-submission item-by-item checklist
  • references/common-issues.md — summary of historical review issues
  • scripts/check_operator.py — automated validation script (25+ check items)
  • scripts/check_overload_consistency.py — multiple-overload operator yaml/mark/op_name consistency check
  • scripts/extract_from_worktree.py — extract 6 PR files from the worktree
  • scripts/gen_pr_description.py — PR description data generation
  • scripts/submit_operator.py — one-stop submission (9 serial steps)
  • scripts/operator_registry.py — canonical name lookup + PR link backfill
  • scripts/pr_gate_check.sh — pre-git push gate (PreToolUse hook)

Overview of check_operator.py automated check items

Check Level
Kernel file exists + KernelGen first line error
No print(), no duplicate functions error
ops/init.py registration + alphabetical order error
_FULL_CONFIG registration (mapped to wrapper) error
Fallback recursion detection (torch.() prohibited in kernel) error
operators.yaml completeness + uniqueness error
Test pytest mark + import method + gems_assert error
gems_assert_close has no rtol error
dtype hardcoding check (needs a constant or a comment) error
Private API torch._xxx error
Benchmark pytest mark + op_name + dtype error
Git commit has no Co-Authored-By error
Inplace mark guard error
yaml/config consistency cross-validation error
Anti-hack Layer 1 (AST) + Layer 2 (dual execution) error
Single-operator PR (git diff validation) error
@use_tl_extra unused stub function error
Hardcoded hyperparameters without comments error
Upstream conflict check error
Test function naming conventions warning
logging module, torch import warning
Code quality (line length, EOF, trailing whitespace) warning
Alias wrapping, NaN handling, Worktree consistency, Wrapper dtype assert warning
Benchmark case count info
.is_cuda / device.type == "cuda" Hardcoded device determination error
Autotune configs hardcoded inline (should be in config file) error
Logger format does not conform to "GEMS <OP_NAME_UPPER>" error
Benchmark torch/gems measurement scope inconsistent error
Test file or Benchmark file missing error
Fused operators placed in ops/ instead of fused/ directory error
Custom shapes duplicate those in core_shapes.yaml warning

Rules requiring manual model inspection (cannot be automated)

The following rules rely on domain judgment, and the model must confirm each item before every submission:

  • dtype pre-check — Does the PyTorch reference implementation of this operator support Half/BFloat16? linalg/special/cdist etc. usually do not
  • benchmark class selection — Does a non-pointwise operator use the correct wrapper class? When custom shapes are needed, has set_shapes？
  • Dead branch cleanup — When pointwise_dynamic already handles type dispatch, are there redundant isinstance checks?
  • Do not modify existing upstream tests — When adding an operator, were the tests of other operators in the same file inadvertently changed?
  • Probabilistic operators — If it is dropout/bernoulli/rand etc., do the tests use statistical methods (mean/variance) for verification rather than exact comparison?
  • Generic first, then specialized — Does the specialized version depend on a generic version that has not yet been merged?
  • Device detection must not use is_cuda — In kernel/wrapper, is .is_cuda or device.type == "cuda"used? Must use flag_gems.device
  • Autotune config location — Are autotune configs placed in the config file? Hardcoding inline within the kernel is prohibited
  • Logger format — logger.debug Does it use the "GEMS <OP_NAME_UPPER>" format?
  • Benchmark fairness — Do torch and gems measure the same scope of computation?
  • Fused operator directory — Are fused operators (AddRMSNorm, FusedRoPE, etc.) placed in src/flag_gems/fused/ instead of ops/？
  • Benchmark shapes deduplication — Do custom shapes duplicate existing entries in core_shapes.yaml ?

Enforcement strategy (the model must comply)

  • check_operator warning = error — submit_operator.py Use --strict mode, all warnings are escalated to errors. The model must not manually bypass the script.
  • AI-generated code is untrustworthy — anti-hack Layer 2 (dual-execution) verifies whether the kernel truly uses Triton for computation. If a hack is detected, the PR is intercepted and the operator is not submitted.
  • Automatic exception logging — submit_operator.py of fatal() automatically appends an event to pr状态记录.md, no manual logging required.