SuperAPI AI Directory

jcadcg-experiments

General & other

View repository

Updated 10-16 09:31

About this skill

Experimental Design and Presentation for the Journal of Computer-Aided Design & Computer Graphics

This skill helps build graphics and geometry evidence that can withstand external review for the Journal of Computer-Aided Design & Computer Graphics (JCAD&CG). This journal is a CCF A-class Chinese monthly in the graphics/CAD field, and external reviewers have higher requirements forgraphics evidence(rendering quality, geometric accuracy, visualization effectiveness, user studies) than for pure leaderboard scores: every claim should be tied to verifiable graphics/geometry evidence. For verification see resources/official-source-map.md (2026-07-09)。

1. Start from the research question (RQ)

First break the contributions into testable RQs, for example:

  • RQ1: Under the same patch budget, is the geometric error of this method lower than the strongest baseline?
  • RQ2: Is the rendering quality (temporal stability) better?
  • RQ3: How much does each module (e.g., feature term, penetration detection) contribute? (ablation)
  • RQ4: Is the overhead (time/memory) acceptable?

Each RQ corresponds to a set of experiments and a table/figure, and the conclusions correspond one-to-one with the RQs.

2. Common evaluation metrics in graphics/CAD

Direction Common metrics
Geometry processing Chamfer distance, Hausdorff distance, normal error, volume/area preservation, feature preservation rate
Rendering/image synthesis PSNR, SSIM, LPIPS, FID (generation), temporal stability, subjective realism score
CAD/CAE/CAM Constraint satisfaction, simulation accuracy, manufacturability, parameter sensitivity
Visualization/visual analytics Task completion time/accuracy, scalability, cognitive load (e.g., NASA-TLX)
VR/AR/MR Latency, frame rate, immersion/comfort scales (e.g., SSQ motion sickness), interaction efficiency
Image and video processing Quantitative metrics + visual comparison, robustness

When reporting metricsspecify the computation script, mask, and alignment method, to avoid incomparability caused by inconsistent protocols.

3. Discipline for fair comparison

  • Same budget/same settings: patch budget, sampling count, compute, training data, and resolution are consistent; otherwise note and explain.
  • Strongest available baseline: include strong methods named in related work, do not avoid them, and do not compare only against weak baselines.
  • Multiple runs: for methods involving randomness, report mean ± standard deviation (e.g., 5 runs), and state the seeds.
  • Statistical significance: when applicable, use paired tests (e.g., p<0.05), especially for user studies.

4. Presentation of graphics evidence (especially important for this journal)

  • Rendering comparison figures: show this method/baseline/GT side by side, with unified viewpoint, lighting, resolution, and cropping; annotate key difference regions (zoom boxes).
  • Geometric error color maps: use a colormap to visualize per-point error, and give the color scale range and units.
  • Ablation charts: remove modules one by one and quantify their contributions.
  • Failure cases: honestly show limitations (e.g., extreme thin walls, anomalous inputs) to enhance credibility.
  • Demo video: Supplement static figures with videos for dynamic results (animation/interaction/timing) to demonstrate stability that static figures cannot capture.

V. User Studies (common in visualization/VR/interaction)

  • DescriptionNumber of participants, recruitment, task design, independent/within-subject design, scales。
  • Report statistical criteria and significance; submit de-identified materials (questionnaires) as supplementary materials.
  • Comply with ethics and informed consent; do not disclose participant identities.

VI. Experiment Self-Check List

  • Each RQ has a corresponding experiment and conclusion, in one-to-one correspondence.
  • Metrics are chosen to fit the direction (geometry/rendering/visualization/interaction), and computation scripts are reproducible.
  • Baselines are the strongest available, with the same budget; differences are noted.
  • For stochastic methods, report mean ± standard deviation and seeds; include significance tests when applicable.
  • Includes rendering comparisons + geometry error heatmaps + ablations + overhead (time/memory).
  • Honestly show failure cases and limitations.
  • User studies (if any) have adequate participant numbers/design/statistics/ethics.

VII. Comparison of Common Experimental Flaws

Flaw Consequence Fix
Only metrics, no visual comparison Reviewers question graphics quality Add rendering comparisons and error heatmaps
Unfair baseline (different budgets) Conclusions are overturned Unify budgets or state clearly
Report the best of a single run Not credible Report mean ± standard deviation over multiple runs
Missing ablations Unclear attribution of contributions Module-by-module ablations
No overhead reported Practicality is questioned Report time/memory/scalability
Avoiding failure cases Appears to be hiding something Honestly present and explain

VIII. Selection of Datasets and Benchmarks

The persuasiveness of graphics/CAD experiments depends largely on whether the data and benchmarks are representative:

  • Cover diversity: For geometry processing, choose models with different topologies/scales/noise; for rendering, choose different materials/lighting/scenes; for CAD, choose parts/assemblies of different complexity.
  • Prefer publicly available: Use publicly recognized public datasets/model repositories as much as possible to facilitate comparison by others; for self-built data, explain how they were constructed and make them public when possible.
  • Appropriate scale: The number of models/scenes should be sufficient to support statistical conclusions; avoid "three models look good".
  • Hard case coverage: Include known difficult situations (thin walls, sharp features, occlusion, high noise) to test robustness.

IX. Standards for Presenting Result Tables and Figures

Presentation Standard
Quantitative table Bold the best value, and the second best may be marked; use the same criteria within a column; give mean ± standard deviation and sample size
Rendering comparison figure Unify viewpoint/lighting/resolution/cropping; label method names and zoom-in boxes for key differences
Geometry error heatmap Use a unified colormap and color scale range; label units; ensure distinguishability in black and white
Ablation figure Compare by removing each module; present quantitatively using the same metric
Overhead figure Time/memory scaling curves; annotate hardware

10. Bind claims to evidence (ask yourself three questions)

  1. For each of my contribution claims, is therea specific table or figurethat directly supports it?
  2. Is the setup of thesupporting evidence fair(same budget, strongest baseline, same data split)?
  3. If a reviewer questions it, can I respond withnew graphical evidence(rather than verbal argument)? (See jcadcg-author-response)

11. Experimental ethics and integrity

  • No selective reporting (cherry-picking): show representative examples rather than only the best ones; report failure cases honestly.
  • Do not hide unfavorable metrics (e.g., high overhead, worse performance under some settings); discuss them honestly in the limitations.
  • User studies must comply with informed consent and de-identification; statistical methods and multiple-comparison corrections must be stated honestly.
  • Metric computation scripts and data are provided with the reproducibility package to ensure verifiability (see jcadcg-reproducibility)。

12. Output format

[JCAD&CG 实验] 就绪 / 需加强
[RQ→证据] RQ1__→表/图__;RQ2__→__;RQ3(消融)__→__;RQ4(开销)__→__
[指标] 几何:__ 渲染:__ 可视化/交互:__(计算脚本可复现?是/否)
[公平性] 相同预算?最强基线?多次运行±std?显著性检验?(是/否)
[图形证据] 渲染对比/误差色图/消融/失败案例/演示视频(齐否)
[用户研究] 人数__ 设计__ 统计__ 伦理__(若无标注 N/A)
[待办] <逐条列出需补实验/证据>