FigureWeave

FigureWeave is a research and engineering project whose goal is to transform paper method description text into publication-style scientific figures while keeping the results as editable SVG. The project is inspired by AutoFigure but has been redone as a more practical figure creation pipeline.

Use FigureWeave is especially suitable for: method overviews, flowcharts, system diagrams, architecture diagrams, and editable sketches for papers, slides, and reports. It is not intended to replace plotting tools driven by precise numerical data, such as matplotlib, seaborn, ggplot, or Origin.

Main capabilities

1. Local GPU SAM3 segmentation: segmentation can run locally on CUDA rather than relying only on a hosted API, thereby improving the speed, privacy, and reproducibility of the icon region extraction stage. Using local SAM3 requires an available NVIDIA GPU, CUDA-enabled PyTorch in the current environment, and Hugging Face access to SAM3. If local SAM3 is unavailable, the codebase can still fall back to other segmentation paths depending on the configuration.
2. Dual-provider model routing: image drafting and SVG inference are decoupled, and the pipeline can use different providers for different stages, for example Gemini -> Gemini, OpenAI -> OpenAI, Gemini -> Anthropic Claude, or OpenAI -> Anthropic Claude.
3. Multi-candidate generation: a single run can generate multiple complete candidates, retain each artifact bundle, write a candidate manifest, and promote the selected result to the default output.
4. Caption-conditioned constraints: the system accepts both method text and captions/figure descriptions, enabling the generator and reconstructor to be constrained by explicit stage structure, layout intent, and narrative emphasis.
5. SVG-first reconstruction pipeline: instead of treating bitmaps as the final result, it explicitly reconstructs an editable SVG template, optionally optimizes the template, and finally assembles the final SVG using the extracted assets.
6. CUDA-accelerated local post-processing: background removal and other local visual post-processing stages use GPU-enabled PyTorch when available, reducing the CPU bottlenecks of the original workflow.
7. More robust fallback behavior: the current pipeline includes explicit fallback paths for no-icon cases, placeholder reduction, and provider-side failures, making batch generation better suited to real-world paper figure drafting.

Usage The project is no longer organized as a single large script: figureweave.py is now a thin compatibility entry point for CLI execution and top-level imports; src/figureweave/config.py stores provider defaults, paths, and shared constants; src/figureweave/llm.py contains Gemini, OpenAI, Claude, OpenRouter, and related model invocation logic; src/figureweave/vision.py covers image drafting, SAM3 segmentation, and background removal; src/figureweave/svg_ops.py handles SVG reconstruction, repair, optimization, and asset replacement; src/figureweave/pipeline.py orchestrates the end-to-end pipeline and multi-candidate execution; src/figureweave/cli.py defines the command-line interface.

FigureWeave is currently divided into five main stages:

1. Image drafting: generate scientific-style sketches based on method text, optional captions, and optional reference images.
2. Segmentation: run local SAM3 or an API backend to detect icons and visual regions, generating samed.png and boxlib.json.
3. Asset extraction: crop the detected regions and remove the background, generating transparent assets.
4. SVG inference and reconstruction: use multimodal models to reconstruct the draft into an editable SVG template, followed by optional optimization.
5. Assembly: replace placeholders with the extracted assets, outputting template.svg, optimized_template.svg, final.svg.

Regarding provider labels, Anthropic Claude is used in this project for understanding and reconstruction, not native image generation; the image drafting stage should use Gemini or OpenAI. If you want to use the same provider for both stages, a single-provider fallback can still be used.

The relevant instructions for starting the server and for the main configuration page and canvas page are presented in the official materials as commands, interface elements, and artifact paths, and some specific commands and configuration items are not fully expanded in the materials. Typical output includes candidates_manifest.json; when multi-candidate mode is enabled, each run is stored in the corresponding directory.

Use cases Suitable for method overviews, flowcharts, system diagrams, architecture diagrams, and editable sketches for papers, slides, and reports.

Limitations This project is not suitable for charts driven by precise numerical data; such tasks should still use tools such as matplotlib, seaborn, ggplot, or Origin. Local SAM3 depends on an NVIDIA GPU, CUDA-enabled PyTorch, and Hugging Face access to SAM3; when unavailable, a fallback path must be relied upon. The specific startup commands, environment variable names, interface operation details, and some configuration items are not fully given in the provided materials.