Cosmos3 Inference
When to use this skill
- Use when a user wants to generate images or videos with Cosmos3
- Use when a user asks about inference parameters, input formats, or parallelism
- Use when a user wants to set up online serving (Ray Serve, Gradio)
- Use when a user asks about prompt engineering or upsampling
- For environment or import errors, hand off to cosmos3-env-troubleshoot
Path convention
All paths below are relative to the cosmos3 package root (../../../ from this skill file). All uv run / python commands should also be run from there.
Where to find answers
| User question |
Go to |
| How do I run inference? (single-GPU, multi-GPU) |
README.md § Inference |
| Which model should I use? (Nano vs Super, memory, shift) |
README.md § Models |
| Which modality? (t2i, t2v, i2v, examples) |
README.md § Modalities |
| What parallelism preset? (latency vs throughput) |
README.md § Inference |
| What input fields are available? (prompt, vision_path, num_frames, ...) |
docs/inference.md § Sample Arguments |
| What are the default parameter values? |
cosmos_framework/inference/defaults/<model_mode>/sample_args.json (per-modality JSON) |
| How do I use custom defaults? |
docs/inference.md § Custom Defaults |
| How do I override a parameter? (precedence) |
docs/faq.md § How do I override a default parameter? |
| What is the shift parameter? |
docs/faq.md § What is the shift parameter? |
| How many frames can I generate? (resolution caps) |
docs/faq.md § How many frames can I generate? |
| How do I start Ray Serve / Gradio / submit requests? |
docs/faq.md § How do I run online inference with Ray? |
| How do I upsample short prompts? |
docs/faq.md § Prompt upsampling |
| How do I use the low-level API? (examples/) |
examples/inference.py (model API) / examples/inference_pipeline.py (pipeline API) |
| All CLI flags |
uv run --all-extras --group=cu130 python -m cosmos_framework.scripts.inference --help |
Things not obvious from the docs
- Path resolution: relative paths in input JSON files are resolved relative to the JSON file's directory, not the working directory.
- Seed: always pass
--seed for reproducible results. Without it, a random seed is used each time.
- Resume: interrupted runs can be resumed by re-running the same command — existing outputs are skipped automatically.
--keep-going: continues processing remaining samples after a per-sample failure (e.g. guardrail rejection). Used in online serving by default.- Unique names: every sample in a run must have a unique
name field, or the script will error.
Related skills
| Skill |
When to use |
../cosmos3-setup/SKILL.md |
Installation and environment setup |
../cosmos3-codebase-nav/SKILL.md |
Finding files, parameters, and configs in code |
../cosmos3-env-troubleshoot/SKILL.md |
Debugging environment and runtime errors |
