openai-evals-tuning

Directory Structure

skills/openai-evals-tuning/
  SKILL.md
  references/
    evals/
      README.md
      working-with-evals.md
      graders.md
      getting-started-datasets.md
      agent-evals.md
      trace-grading.md
      external-models.md
      evaluation-best-practices.md
    fine-tuning/
      README.md
      model-optimization.md
      supervised-fine-tuning.md
      vision-fine-tuning.md
      direct-preference-optimization.md
      reinforcement-fine-tuning.md
      reinforcement-fine-tuning-use-cases.md
      fine-tuning-best-practices.md
      prompt-optimizer.md
  samples/
    README.md
    eval-create-and-run.md
    grader-definitions.md
    sft-dataset-and-job.md
    dpo-basics.md
    rft-basics.md
    distillation-workflow.md
  scripts/
    README.md
    fine-tuning-jobs.md
    training-data-files.md
    evals.md

Search Procedure

Look up the category from the task, and identify the target page in the category's README.md:

1. Find the category corresponding to the task in the mapping table below
2. of that category references/{category}/README.md refer to and identify the target page
3. Read the .md of the relevant page to check the details

Task → Category mapping

Task	Category	Reference README
I want to create and run an eval with the Evals API	evals	references/evals/README.md
I want to define a grader (string check / text similarity / model / python / multi)	evals	references/evals/README.md
I want to score and evaluate an agent's trace	evals	references/evals/README.md
I want to evaluate third-party models and external endpoints	evals	references/evals/README.md
I want to create a fine-tuning job with SFT / DPO / RFT	fine-tuning	references/fine-tuning/README.md
I want to perform vision fine-tuning or distillation	fine-tuning	references/fine-tuning/README.md
I want to know how to use the prompt optimizer	fine-tuning	references/fine-tuning/README.md
I want to know typical usage	samples	samples/README.md
I want to operate jobs, files, and evals with curl	scripts	scripts/README.md

Notes

• Self-service fine-tuning (SFT / DPO / RFT / vision fine-tuning / distillation) is being gradually deprecated. It cannot be used in new organizations after 2026-05-07, and existing organizations will also be unable to create new jobs after 2027-01-06. The prompt optimizer is scheduled to be discontinued on 2026-11-30. Individual deadlines are listed in the ## Notes of each page
• How to use reasoning models and the reference for the Responses API itself are handled by the openai-api-core skill. This skill is limited to evaluation (evals) and model optimization (fine-tuning)