Workflow
- Determine the endpoint type and model capabilities, and confirm whether the SDK should really be used instead of the Proxy.
- Check the current official LiteLLM documentation and provider support status; do not guess model names or parameters from memory.
- Read credentials from environment variables and specify a provider-prefixed model.
- Implement the success path and the authentication, rate limit, timeout, and provider error paths.
- Validate structured output again at the application layer; for tool calling, only execute registered tools and validate arguments.
- Streaming must handle empty deltas, mid-stream failures, and cancellation.
- Run offline tests; only call paid providers when the user explicitly allows it.
When you need code examples, test cases, or version notes, read Experiments and references。
Security bottom line
- Do not hard-code or output real API keys.
- Do not arbitrarily execute function names or unvalidated arguments generated by the model.
- Do not assume that all models support the same tools, JSON mode, reasoning, or streaming behavior.
- Cost data is an estimate; when you need prices, re-check the official model cost map.
Acceptance
- The minimal success case can be run or verified by a mock.
- The three main error categories have explicit handling and tests.
- Structured output has a schema and application-layer validation.
- Tool calling adopts a rejection policy for unknown tools and invalid arguments.
- Report the execution command, expected results, and actual evidence.