famou-artifact-generator

FaMou evolution task problem definition

End-to-end guidance: from a vague idea → task specification document problem.md → implementation and verification of the evaluator and initial solution.

Two-phase process:

• Phase 1 (Clarification): through structured Q&A, then build and produce the task specification document problem.md
• Phase 2 (Implementation): based on problem.md generate and verify init.py、evaluator.py、prompt.md

The three inputs of a FaMou evolution task:

• init.py — candidate solution script (single file, directly runnable)
• evaluator.py — evaluation logic (fixed interface, see below)
• prompt.md — evolution guidance (role, task, data, solution reference; ≤100 lines)

---

Phase 1: Clarification phase

Step 1 — Understand the current state of the project

• Check data files, code scripts, README, configuration; for questions that can be answered from them,do not ask again。
• If there is data, first analyze and understand the data, such as the data format, possible problem descriptions, etc.

---

Step 2 — Information gap analysis

Identify all the information needed to fill in the task template, and use the Plan or TODO tool to create a TODO list:

Core guidelines

1. Items that can be answered from documents/data should be marked as resolved directly, and not asked again.
2. If the answer to an item is relatively clear, or it is not a key question, mark it as resolved directly.

Required TODOs(resolve one by one in dependency order; do not skip or reorder):

• Core problem description and optimization objective
• Input/output specification (command-line arguments, data files, standard output/generated files)
• Data source (path, structure/schema, field definitions, access method)
• Various constraints and their verification methods
• Evaluator scoring logic (how to distinguish different quality levels)
• Initial solution approach or baseline strategy

Optional TODOs(fill in if present, skip if not):

• Supplementary reference information

---

Step 3 — Active clarification loop

Repeat until all required TODOs are resolved:

1. Select an unresolved required TODO.
2. Construct clarification questions to help the model collect the necessary contextual information (use ask_uer type tools):
   • Specifysingle choiceormultiple choice。
   • Provide at least 3 options, and the last item is always **"Other (please specify)"**.
3. Wait for user input.
4. Validate the answer:
   • If invalid or irrelevant, prompt the user to choose again, and return to Step 3
   • If valid, record the information and proceed to Step 5.
5. Determine whether the information for the current TODO is sufficient:
   • Sufficient → mark the TODO as resolved, and return to Step 1.
   • Insufficient(especially hard constraints, soft constraints and other information-rich items) → continue constructing follow-up questions for the remaining information gaps, return to step 2,do not skip to the next TODO。

After all required TODOs are resolved, or the user explicitly confirms that the task is clear, exit the loop.

Constraint-type TODO clarification guidance: Constraint clarification usually involves multiple dimensions; ask only one dimension per round, collect them completely step by step, and then mark it complete.

---

Step 4 — Synthesize the specification and write it into problem.md

According to thetask templatebelow, fill in all sections, rules:

• The explicitly clarified information overrides the existing problem.md content.
• If problem.md already exists, merge and fully overwrite it when writing.
• Keep it concise, no redundancy.

---

Step 5 — Review and confirmation

"Please review problem.md. Is the problem definition now clear and complete?"

• Yes → ask whether to enter the implementation phase, and if confirmed, enterimplementation phase。
• No → return to step 2 (only when new gaps can be identified).

---

Phase 2: Implementation

Based on the confirmed problem.md, generate and verify the evaluator.py、init.py and prompt.md。

---

Step 6 — Artifact implementation

evaluator.py(implement first, because init.py verification depends on it)

• Strictly follow the interface definition below (READONLY).
• Strictly follow the system-level constraints (READONLY).
• Scoring must be able to distinguish quality levels: crash/violate constraints → 0; low quality → low score; high quality → high score.

init.py

• Single file, directly runnable, satisfies all hard constraints.
• Implement problem.md the baseline strategy or initial solution direction specified in.

prompt.md(≤100 lines)

• Contains four parts: (1) role, (2) task description, (3) data description, (4) feasible solution reference.
• The first three parts come from problem.md, and no new assumptions may be added.
• The document content must not contain any emoji.

---

Step 7 — Verification

Run the evaluator to verify init.py：

python evaluator.py <path_to_init.py>

Must simultaneously satisfy:

• validity == 1
• combined_score is numeric and != 0
• error_info == ""

If any check fails, analyze the cause, modify the corresponding artifact, and verify again until all pass.

---

Task template (problem.md structure)

## 目标
<!-- READONLY：为 FaMou 进化任务准备 init.py、evaluator.py 和 prompt.md -->

## 1. 任务定义
- 核心问题描述
- 输入：（命令行参数 / 数据文件）
- 输出：（标准输出 / 生成文件）
- 主要优化目标
- 关键指标及计算公式

## 2. 数据描述
- 数据来源（文件/表名、结构/模式、字段定义）
- 访问方式（路径、格式）

## 3. 评估器

### 接口（READONLY）
def evaluate(path_user_py: str, task_name: str = "default", timeout: int = 3600) -> dict:
    return {
        "validity": float,        # 0 或 1；1 表示满足所有硬约束
        "combined_score": float,  # 为可行解打分，0表示无效解，低分表示低质量解，高分表示高质量解
        "cost_time": float,       # 评估花费时间
        "error_info": str,        # 成功时为 ""
    }

### 参考执行流程
1. 创建所需中间目录和文件
2. 配置候选解运行参数
3. 执行候选解脚本
4. 捕获执行结果（完成 / 错误 / 异常）
5. 检查硬约束和软约束
6. 计算 combined_score

### 硬约束
<!-- 精确规则、验证方法、违反后果 -->

### 软约束
<!-- 优化目标、度量方法、对分数影响 -->

### 系统级约束（READONLY）
- 无论 init.py 是否与 evaluator.py 同目录，评估器必须能正确执行
- 如果使用 subprocess 调用子进程，必须将 cwd 设置为 evaluator.py 所在目录
- 不得依赖临时目录作为工作目录

## 4. 初始解
<!-- 单文件 init.py；方案、基线策略或求解方向；必须满足所有硬约束 -->

## 5. 进化提示（prompt.md，≤100 行）
<!-- (1) 角色 (2) 任务描述 (3) 数据描述 (4) 可行解参考 -->

## 6. 补充信息（可选）
<!-- 参考资料、备注 -->

---

Notes

• The clarification loop asks only one question per round, and questions must not be asked in parallel.
• Invalid answers must be rejected, and unresolved TODOs must not be assumed.
• The implementation phase must run verification and must not skip it.
• If the user wants to skip clarification and implement directly, respect their choice, mark the known information gaps in problem.md as assumptions, and then directly enter Phase 2.