Milk infrastructure is a set of infrastructure for deploying, managing, and scaling AI model calls on any cloud. It consists of two core components: the gateway Milk Parlor and the agent Milk Man.
Milk Parlor is a gateway compatible with the OpenAI SDK, responsible for forwarding model calls and returning answers, while in the background saving qualifying request-response pairs. Developers only need to keep their existing OpenAI SDK and calling method, changing the URL and key to integrate; the application receives answers as usual, while Milk makes use of the saved traffic.
Milk Man is an agent. Give it a goal, and it will execute commands, launch cloud models, measure results, and save what it learns. The tasks it can perform include: running models, comparing different settings of models, or training based on saved data. It continuously runs jobs and retains results; closing the dashboard will not stop it.
Milk's workflow is divided into five stages:
01 Collect: Save each qualifying request and response with an account-unique ID. Related interactions are grouped into one task.
02 Understand: Count the number of new interactions, and when configured milestones are reached, save summaries about topics, tasks, and response times. Summary milestones are set via the environment variable MILK_SUMMARY_THRESHOLDS; one interaction refers to one request and its response.
03 Generate examples: A teacher model writes new tasks based on this traffic. Training tasks remain separate from the tasks used to judge improvement.
04 Train and compare: Train an open model, adjust how it runs, and compare answers, speed, and cost on the same tasks. Qwen3.5-0.8B is the first training base.
05 Review and route: Prepare a gateway rule for the selected model; only after an operator reviews and signs off will customer traffic switch over.
Regarding storage and compute configuration, environment variables can choose local files or S3-compatible object storage, hosted inference APIs, and your own Baseten or Modal GPUs. Model files and runtime images are cached separately. Milk Man can run locally, while data and compute remain in the cloud.
Regarding goals and evaluation, Milk retains experiments and their results, including failures. Speed improvements and answer quality improvements are two different measurements. Its goal is to beat the selected baseline on your tasks, not to claim some kind of universal ranking.
The interface content displayed on the official website is recorded developmentresults, not real-time status.
