This skill helps build graphics and geometry evidence that can withstand external review for the Journal of Computer-Aided Design & Computer
Graphics (JCAD&CG). This journal is a CCF A-class Chinese monthly in the graphics/CAD field,
and external reviewers have higher requirements forgraphics evidence(rendering quality, geometric accuracy, visualization effectiveness, user studies) than for pure leaderboard scores:
every claim should be tied to verifiable graphics/geometry evidence. For verification see resources/official-source-map.md
(2026-07-09)。
1. Start from the research question (RQ)
First break the contributions into testable RQs, for example:
- RQ1: Under the same patch budget, is the geometric error of this method lower than the strongest baseline?
- RQ2: Is the rendering quality (temporal stability) better?
- RQ3: How much does each module (e.g., feature term, penetration detection) contribute? (ablation)
- RQ4: Is the overhead (time/memory) acceptable?
Each RQ corresponds to a set of experiments and a table/figure, and the conclusions correspond one-to-one with the RQs.
2. Common evaluation metrics in graphics/CAD
| Direction | Common metrics |
|---|---|
| Geometry processing | Chamfer distance, Hausdorff distance, normal error, volume/area preservation, feature preservation rate |
| Rendering/image synthesis | PSNR, SSIM, LPIPS, FID (generation), temporal stability, subjective realism score |
| CAD/CAE/CAM | Constraint satisfaction, simulation accuracy, manufacturability, parameter sensitivity |
| Visualization/visual analytics | Task completion time/accuracy, scalability, cognitive load (e.g., NASA-TLX) |
| VR/AR/MR | Latency, frame rate, immersion/comfort scales (e.g., SSQ motion sickness), interaction efficiency |
| Image and video processing | Quantitative metrics + visual comparison, robustness |
When reporting metricsspecify the computation script, mask, and alignment method, to avoid incomparability caused by inconsistent protocols.
3. Discipline for fair comparison
- Same budget/same settings: patch budget, sampling count, compute, training data, and resolution are consistent; otherwise note and explain.
- Strongest available baseline: include strong methods named in related work, do not avoid them, and do not compare only against weak baselines.
- Multiple runs: for methods involving randomness, report mean ± standard deviation (e.g., 5 runs), and state the seeds.
- Statistical significance: when applicable, use paired tests (e.g., p<0.05), especially for user studies.
4. Presentation of graphics evidence (especially important for this journal)
- Rendering comparison figures: show this method/baseline/GT side by side, with unified viewpoint, lighting, resolution, and cropping; annotate key difference regions (zoom boxes).
- Geometric error color maps: use a colormap to visualize per-point error, and give the color scale range and units.
- Ablation charts: remove modules one by one and quantify their contributions.
- Failure cases: honestly show limitations (e.g., extreme thin walls, anomalous inputs) to enhance credibility.
- Demo video: Supplement static figures with videos for dynamic results (animation/interaction/timing) to demonstrate stability that static figures cannot capture.
V. User Studies (common in visualization/VR/interaction)
- DescriptionNumber of participants, recruitment, task design, independent/within-subject design, scales。
- Report statistical criteria and significance; submit de-identified materials (questionnaires) as supplementary materials.
- Comply with ethics and informed consent; do not disclose participant identities.
VI. Experiment Self-Check List
- Each RQ has a corresponding experiment and conclusion, in one-to-one correspondence.
- Metrics are chosen to fit the direction (geometry/rendering/visualization/interaction), and computation scripts are reproducible.
- Baselines are the strongest available, with the same budget; differences are noted.
- For stochastic methods, report mean ± standard deviation and seeds; include significance tests when applicable.
- Includes rendering comparisons + geometry error heatmaps + ablations + overhead (time/memory).
- Honestly show failure cases and limitations.
- User studies (if any) have adequate participant numbers/design/statistics/ethics.
VII. Comparison of Common Experimental Flaws
| Flaw | Consequence | Fix |
|---|---|---|
| Only metrics, no visual comparison | Reviewers question graphics quality | Add rendering comparisons and error heatmaps |
| Unfair baseline (different budgets) | Conclusions are overturned | Unify budgets or state clearly |
| Report the best of a single run | Not credible | Report mean ± standard deviation over multiple runs |
| Missing ablations | Unclear attribution of contributions | Module-by-module ablations |
| No overhead reported | Practicality is questioned | Report time/memory/scalability |
| Avoiding failure cases | Appears to be hiding something | Honestly present and explain |
VIII. Selection of Datasets and Benchmarks
The persuasiveness of graphics/CAD experiments depends largely on whether the data and benchmarks are representative:
- Cover diversity: For geometry processing, choose models with different topologies/scales/noise; for rendering, choose different materials/lighting/scenes; for CAD, choose parts/assemblies of different complexity.
- Prefer publicly available: Use publicly recognized public datasets/model repositories as much as possible to facilitate comparison by others; for self-built data, explain how they were constructed and make them public when possible.
- Appropriate scale: The number of models/scenes should be sufficient to support statistical conclusions; avoid "three models look good".
- Hard case coverage: Include known difficult situations (thin walls, sharp features, occlusion, high noise) to test robustness.
IX. Standards for Presenting Result Tables and Figures
| Presentation | Standard |
|---|---|
| Quantitative table | Bold the best value, and the second best may be marked; use the same criteria within a column; give mean ± standard deviation and sample size |
| Rendering comparison figure | Unify viewpoint/lighting/resolution/cropping; label method names and zoom-in boxes for key differences |
| Geometry error heatmap | Use a unified colormap and color scale range; label units; ensure distinguishability in black and white |
| Ablation figure | Compare by removing each module; present quantitatively using the same metric |
| Overhead figure | Time/memory scaling curves; annotate hardware |
10. Bind claims to evidence (ask yourself three questions)
- For each of my contribution claims, is therea specific table or figurethat directly supports it?
- Is the setup of thesupporting evidence fair(same budget, strongest baseline, same data split)?
- If a reviewer questions it, can I respond withnew graphical evidence(rather than verbal argument)? (See
jcadcg-author-response)
11. Experimental ethics and integrity
- No selective reporting (cherry-picking): show representative examples rather than only the best ones; report failure cases honestly.
- Do not hide unfavorable metrics (e.g., high overhead, worse performance under some settings); discuss them honestly in the limitations.
- User studies must comply with informed consent and de-identification; statistical methods and multiple-comparison corrections must be stated honestly.
- Metric computation scripts and data are provided with the reproducibility package to ensure verifiability (see
jcadcg-reproducibility)。
12. Output format
[JCAD&CG 实验] 就绪 / 需加强
[RQ→证据] RQ1__→表/图__;RQ2__→__;RQ3(消融)__→__;RQ4(开销)__→__
[指标] 几何:__ 渲染:__ 可视化/交互:__(计算脚本可复现?是/否)
[公平性] 相同预算?最强基线?多次运行±std?显著性检验?(是/否)
[图形证据] 渲染对比/误差色图/消融/失败案例/演示视频(齐否)
[用户研究] 人数__ 设计__ 统计__ 伦理__(若无标注 N/A)
[待办] <逐条列出需补实验/证据>