famou-experiment-manager

Famou Task Management Skill

Complete workflow for submitting and managing experiment tasks via famou-ctl-sdk .

Prerequisite Environment Check

1.1 Check whether famou-ctl-sdk is installed

famou-ctl --version

If the version number is output, continue to the next step
If the command is not found, run the installation:pip install famou-sdk --pre

After installation is complete, verify famou-ctl --versionagain; if it still fails, stop and prompt the user to check the Python environment or pip source configuration.

1.2 Check and configure API configuration

Use the helper script scripts/config.py to check and configure API settings.

Check API configuration

python3 scripts/config.py read

Check Result	Action
`status: "ok"`	Configuration complete, skip configuration
`status: "missing"`	Execute API Key configuration

Configure API:

Prompt the user to enter a valid API_KEY, then execute the configuration command

python3 scripts/config.py write <YOUR_API_KEY>

Submit FaMou Experiment

2.1 Find the config.yaml file

Recursively find all config.yaml files in the current working directory:

find . -name "config.yaml" -type f 2>/dev/null | sort

Handling results:

Situation	Action
Found 1	Use it directly, inform the user of the path, continue to the next step
Found multiple	Use the `ask_user` tool to ask the user which one to choose
Not found	Report to the user and prompt the user to create config.yaml

config.yaml template:

evolve_config:
  max_iterations: 100
  population_size: 100
  num_islands: 2
initial_program: "init.py"
evaluator: "evaluator.py"
system_message: "prompt.md"

2.2 Confirm the experiment directory

Use the parent directory (absolute path) of the selected config.yaml as the experiment directory:

# 示例：若 config.yaml 路径为 ./experiments/my_exp/config.yaml，则实验目录是 /absolute/path/to/experiments/my_exp
realpath $(dirname <config.yaml路径>)

2.3 Get the experiment name

Use the ask_user tool or directly ask the user in the conversation to enter the experiment name experiment_name, Prompt the user that the experiment name can only contain letters, numbers, and underscores, and must not exceed 20 characters

2.4 Submit the experiment

famou-ctl experiment create \
  --config <config.yaml绝对路径> \
  --experiment-name <experiment_name> \
  --json

Process the output:

Command succeeded: parse the JSON output, display key information such as experiment ID and status
Command failed: display the error message, prompt the user to check the configuration or network connection

2.5 Experiment status query

Every 10s, query the experiment status to check whether it has passed the online validation and normally entered famou evolution (validation takes some time, so polling is needed to check the experiment status)

Validation failed: fix the evaluator and initial solution, delete the failed experiment, and resubmit
Validation succeeded: output the experiment status and end

Other FaMou Experiment Operations

Step 1: Confirm experiment-id

Find the experiment-id in the context and use it directly
If not provided, use the ask_user tool to request user input

Step 2: Execute the corresponding command

famou-ctl experiment status <experiment-id> --json  # 查看实验状态
famou-ctl experiment cancel <experiment-id> --json  # 取消实验
famou-ctl experiment delete <experiment-id> --json  # 删除实验
famou-ctl experiment logs <experiment-id> --follow/-f --output <file-path> --api-url <url> --json  # 查看并保存实验日志
famou-ctl experiment results <experiment-id> --output <file-path> --json  # 查看实验结果