llm-ops

LLM-OPS -- Production AI

Overview

LLM Operations -- RAG, embeddings, vector databases, fine-tuning, advanced prompt engineering, LLM costs, quality evals, and AI architectures for production. Activate for: implementing RAG, creating an embeddings pipeline, Pinecone/Chroma/pgvector, fine-tuning, prompt engineering, LLM cost reduction, evals, semantic cache, streaming, agents.

When to Use This Skill

• When you need specialized assistance with this domain

Do Not Use This Skill When

• The task is unrelated to llm ops
• A simpler, more specific tool can handle the request
• The user needs general-purpose assistance without domain expertise

How It Works

The difference between an AI prototype and an AI product is operability. LLM-Ops is the engineering that makes AI reliable, scalable, and economical.

---

Complete Rag Architecture

[Documents] -> [Chunking] -> [Embeddings] -> [Vector DB] | [Query] -> [Embed query] -> [Semantic Search] -> [Top K chunks] | [LLM + Context] -> [Response]

Indexing Pipeline

from anthropic import Anthropic import chromadb

client = Anthropic()
chroma = chromadb.PersistentClient(path="./chroma_db")

def chunk_text(text, chunk_size=500, overlap=50):
    words = text.split()
    chunks = []
    for i in range(0, len(words), chunk_size - overlap):
        chunk = " ".join(words[i:i + chunk_size])
        if chunk: chunks.append(chunk)
    return chunks

def index_document(doc_id, content_text, metadata=None):
    chunks = chunk_text(content_text)
    ids = [f"{doc_id}_chunk_{i}" for i in range(len(chunks))]
    collection.upsert(ids=ids, documents=chunks)
    return len(chunks)

Query Pipeline With Rag

def rag_query(query, top_k=5, system=None): results = collection.query( query_texts=[query], n_results=top_k, include=["documents", "metadatas", "distances"]) context_parts = [] for doc, meta, dist in zip(results["documents"][0], results["metadatas"][0], results["distances"][0]): if dist < 1.5: src = meta.get("source", "doc") context_parts.append(f"[Source: {src}] {doc}") context = "

---

".join(context_parts) response = client.messages.create( model="claude-opus-4-20250805", max_tokens=1024, system=system or "Respond based on the context.", messages=[{"role": "user", "content": f"Context: {context}

{query}"}]) return response.content[0].text

---

Choosing The Vector Db

DB	Best For	Hosting	Cost
Chroma	Development, local	Self-hosted	Free
pgvector	Already uses PostgreSQL	Self/Cloud	Free
Pinecone	Managed production	Cloud	USD 70+/month
Weaviate	Multi-modal	Self/Cloud	Free+
Qdrant	High performance	Self/Cloud	Free+

Pgvector

CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE knowledge_embeddings ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), content TEXT NOT NULL, embedding vector(1536), metadata JSONB, created_at TIMESTAMPTZ DEFAULT NOW() ); CREATE INDEX ON knowledge_embeddings USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100); SELECT content, 1 - (embedding <=> QUERY_VECTOR) AS similarity FROM knowledge_embeddings ORDER BY similarity DESC LIMIT 5;

---

Elite Prompt Structure

Components of the Auri system prompt:

• Identity: Name (Auri), Tone (Natural, warm, direct), Platform (Amazon Alexa)
• Rules: Maximum 3 short paragraphs, no markdown, conversational language
• Capabilities: business analysis, data-driven advice, creativity
• Limitations: no real-time internet, no financial transactions
• Personalization: {user_name}, {user_preferences}, {relevant_history}

Chain-Of-Thought

def cot_analysis(problem: str) -> str: steps = [ "1. What exactly is being asked?", "2. What information is critical to solve it?", "3. What possible approaches exist?", "4. Which approach is best and why?", "5. What risks or limitations exist?", ] prompt = f"Analyze step by step:

PROBLEM: {problem}

" prompt += " ".join(steps) + "

Final answer (concise, for voice):" return call_claude(prompt)

---

Semantic Cache

class SemanticCache: def init(self, similarity_threshold=0.95): self.threshold = similarity_threshold self.cache = {}

    def get_cached(self, query, embedding):
        for cached_emb, (response, _) in self.cache.items():
            if cosine_similarity(embedding, cached_emb) >= self.threshold:
                return response
        return None

    def set_cache(self, query, embedding, response):
        self.cache[tuple(embedding)] = (response, query)

Claude Cost Estimation

PRICING = { "claude-opus-4-20250805": {"input": 15.00, "output": 75.00}, "claude-sonnet-4-5": {"input": 3.00, "output": 15.00}, "claude-haiku-3-5": {"input": 0.80, "output": 4.00}, }

def estimate_monthly_cost(model, avg_input, avg_output, req_per_day):
    p = PRICING[model]
    daily = (avg_input + avg_output) * req_per_day / 1e6
    monthly = daily * p["input"] * 30
    return {"model": model, "monthly_cost": "USD %.2f" % monthly}

---

Evaluation Framework

from anthropic import Anthropic client = Anthropic()

def evaluate_response(question, expected, actual, criteria):
    criteria_text = "

".join(f"- {c}" for c in criteria) eval_prompt = ( f"Evaluate the AI assistant's response.

" f"QUESTION: {question} EXPECTED ANSWER: {expected} " f"CURRENT ANSWER: {actual}

Criteria: {criteria_text}

" "Score 0-10 and justification for each criterion. JSON format." ) response = client.messages.create( model="claude-haiku-3-5", max_tokens=1024, messages=[{"role": "user", "content": eval_prompt}] ) import json return json.loads(response.content[0].text)

AURI_EVALS = [
    {
        "question": "Quais sao os principais riscos de abrir startup agora?",
        "criteria": ["precisao_factual", "relevancia", "clareza_para_voz"]
    },
]

---

6. Commands

Command	Action
/rag-setup	Configures the complete RAG pipeline
/embed-docs	Indexes documents in the vector DB
/prompt-optimize	Optimizes prompt for quality and cost
/cost-estimate	Estimates the monthly LLM cost
/eval-run	Runs the quality evals suite
/cache-setup	Configures the semantic cache
/model-select	Chooses the ideal model for the use case

Best Practices

• Provide clear, specific context about your project and requirements
• Review all suggestions before applying them to production code
• Combine with other complementary skills for comprehensive analysis

Common Pitfalls

• Using this skill for tasks outside its domain expertise
• Applying recommendations without understanding your specific context
• Not providing enough project context for accurate analysis