Turn "resume + vague role" into abased on real interview experience contentpersonalized prep package. You (the agent) are responsible for reasoning and judgment;scripts/ the Python scripts under it handle the deterministic dirty work. The two interact through the JSON files in corpus_cache/ .
Input
- Resume: PDF, image, or text file path.
- Vague role: a direction, e.g. "AI application development" (nota specific JD).
Tools (run with the package's venv:.venv/bin/python)
scripts/resume_extract.py→extract_resume(path) -> ResumeExtraction{text, needs_vision, asset_path}scripts/connectors/github.py→GithubConnector(repo_raw_urls).search(queries) -> SearchResultscripts/connectors/nowcoder.py→NowCoderConnector(post_urls).search(queries) -> SearchResultscripts/connectors/xiaohongshu.py→XiaohongshuConnector(export_path=..., driver=..., enable_image_ocr=...).search(queries) -> SearchResult(choose one of two:export_pathreads pre-generated JSON;driverautomatically runs MediaCrawler;enable_image_ocr=Trueis deep mode, images are downloaded/OCR'd and used as the main body text;Falseis fast mode, reads only titles, body captions, tags, and timestamps)scripts/scrape/mediacrawler_driver.py→MediaCrawlerDriver(home=None).scrape_xhs(keywords, login_type="qrcode"|"cookie") -> Path,Driver mode——shell out to call the locally installed MediaCrawler (by default found from$MEDIACRAWLER_HOMEor~/.mediacrawler/). Cookie login recommended: setconfig/base_config.pyandLOGIN_TYPE = "cookie"in MediaCrawlerCOOKIES = "web_session=<value>"。scripts/scrape/normalize_xhs.py→normalize(notes) -> list[dict](CLI:python -m scripts.scrape.normalize_xhs <in.json> -o <out.json>), normalize MediaCrawler's native output into the input forXiaohongshuConnector. Only needed for manual mode; in driver mode the connector calls it automatically internally.scripts/ocr/extract.py→extract_text_from_image(path, engine=None, min_confidence=0.6) -> OcrResult{text, confidence, needs_vision}scripts/ocr/xhs_images.py→ download Xiaohongshuimage_list, by default detects RapidOCR, merges paginated OCR intoRawPost.content_text/raw_text; when low quality, setneeds_vision_fallback=Truescripts/corpus/extraction.py→extract_candidates(raw_posts) -> list[ExtractionCandidate]+materialize_questions(candidates, decisions) -> ExtractionMaterializationResult(candidate original text segmentation + constrained decision-based question selection)scripts/corpus/store.py→save_raw_posts / load_raw_posts / save_questions / load_questions / save_extraction_candidates / load_extraction_candidates / save_extraction_decisions / load_extraction_decisionsscripts/corpus/runner.py→run_pipeline(raw_posts_path, decisions_path, output_dir, today=None) -> PipelineRunResult(offline, replayably generates input copies, candidates, selected questions, ranked questions, rejected items, diagnostics, and a SHA-256 manifest)scripts/corpus/recency.py→classify_recency(...)+filter_recent(posts, window_days=730, today=None, keep_undated=True) -> list[RawPost]scripts/corpus/dedupe_rank.py→dedupe_and_rank(questions) -> list[Question](first by number of independent sources, then by occurrence count and recency;question_rank_score()can view scoring details)scripts/corpus/quality.py→prepare_questions(raw_posts, questions, ...) -> QuestionPreparationResult(unified entry point for evidence validation, recency filtering, deduplication and sorting; new runs reject questions without evidence by default)- The data model is in
scripts/models.py; for structural description seeassets/schema.md。
Workflow
-
Preparation (only when the Xiaohongshu source is enabled). Choose one of the two modes, see
docs/setup/mediacrawler.md:- driver mode (recommended): the userone-timeinstalls MediaCrawler and logs in. Prefer cookie mode: copy
web_sessionfrom a normal browser, write it into MediaCrawlerconfig.COOKIES, then useXiaohongshuConnector(driver=MediaCrawlerDriver(), login_type="cookie")to run collection automatically. QR code mode is still available, but more likely to trigger risk control. - Manual mode: the user runs MediaCrawler +
normalize_xhs.pythemselves each time, and feedscorpus_cache/xhs_export.jsontoXiaohongshuConnector(export_path=...)。 - Reading depth must be explained: many Xiaohongshu interview experiences are limited by the text area, and the complete questions are in the images.
fastmode reads only titles/body/tags, suitable for verifying recall; for the formal prep package preferdeepmode (enable_image_ocr=True) to read image OCR. If usingfast, you must explicitly write in the output "Image OCR not read; may have missed complete questions in the image". Text/Nowcoder/GitHub sources do not need this step.
- driver mode (recommended): the userone-timeinstalls MediaCrawler and logs in. Prefer cookie mode: copy
-
Resume understanding. Call
extract_resume. Ifneeds_visionis true, use your own vision capability to directly read this image/PDF. Produce a structured summary: skills, projects (technologies used in each project), key terms. -
Seed query generation. Use yourown domain knowledge, derive seed queries from "job direction + resume". This is domain-agnostic: no matter what the domain (marketing, quant, backend, design...), you already know the relevant job aliases and underlying skills/topics, so just generate them on the spot.Do not rely on any preset word list. Seeds come from two places: (a) relevant job aliases implied by the job direction; (b) specific skills/projects/keywords extracted from the resume. Prefer underlying skill/topic terms rather than job titles—they are more stable and have better recall.
-
Iterative retrieval. Source priority:Nowcoder + Xiaohongshu (primary, with timestamps) > GitHub (supplementary, often outdated)。
3a. URL discovery (do this first each round). Use your search capability (WebSearch or equivalent tool) to run the current seed queries once, and collect candidate URLs. Bucket by domain:
nowcoder.com/discuss/<post_id>→ goes toNowCoderConnector(post_urls=...)xiaohongshu.com/explore/<note_id>→ cannot be scraped directly; if the Xiaohongshu source is enabled, throw the current keywords toXiaohongshuConnector(driver=MediaCrawlerDriver()).search([keywords]), which will automatically shell out to run MediaCrawler. If not enabled, note down the note ID and let the user configure it according todocs/setup/mediacrawler.mdorgithub.com/<owner>/<repo>/blob/<branch>/<path>→ convert to raw URL → goes toraw.githubusercontent.com/...Other public content pagesGithubConnector(repo_raw_urls=...)- (Zhihu articles, CSDN articles, personal blogs, woshipm/uisdc, etc.): use WebFetch to pull back the body text, manually construct, and
RawPost(source="webfetch:<domain>", post_type="text", raw_text=<正文>, posted_at=<页面可见日期或 None>)together with connector resultssave_raw_posts
Explicitly exclude: aggregation/listing pages (only accept article pages); 403/login-required pages (do not waste fetch quota, note them down as knowledge gaps and tell the user).
3b. Call connectors + harvest. Feed the bucketed URLs to the corresponding connector, and use
save_raw_poststo persist the results to disk. Read the results,harvest job titles / tags / high-frequency terms that actually appear, and use the harvested terms to run the next round of 3a until no new terms emerge. If some connector returnsstatus="degraded"(for example, Nowcoder needs a cookie, Xiaohongshu needs MediaCrawler to be run first, or the message containsselectorindicating HTML drift), tell the user what it needs; degradation of primary sources will significantly affect timeliness, so you must explicitly warn the user—do not silently make do with GitHub.GitHub calls must always include
relevance_hints(do not passNoneor[])。 GitHub repositories contain a lot of algorithm/interview-cram material; without filtering, it will severely pollute the corpus.GithubConnector(repo_raw_urls=..., relevance_hints=<当前一轮的术语/岗位别名>)。Cold start(when there are no harvested results yet in the first round), directly pass the seed queries from step 2 as hints—at all times, pass a non-empty list. Match rule: substring, case-insensitive; keep it as long as any hint appears in the body text.Human-in-the-loop: Before the final round, show the user the directions/terms you discovered from real data, and let them add/remove/correct.
-
Content-level relevance judgment. throughRead post contentJudge whether each one is relevant against the user's target role + resume—notby whether the post's job title matches some preset table.
-
Candidate extraction and semantic decisions. For text-type RawPost, directly use
content_text/raw_text。Image-type RawPost (Xiaohongshu,post_type="image"): the formal package uses deep mode by default to readcontent_text/raw_text, which preferentially comes from image OCR; title, caption, and tags are only used inlocator_textfor recall/localization. Ifneeds_vision_fallback=True, first follow theasset_pathsorder to supplement-read images, and write the visually supplement-readverbatim original textinto that post'scontent_text(preserving existing OCR text), then continue extracting questions. If fast mode was used this time for speed (enable_image_ocr=False), the title/body/tags can only be used as clues, and you must not claim to have covered the complete interview-experience questions in the images.For all
RawPostcallextract_candidates(raw_posts), and usesave_extraction_candidatesto write intocorpus_cache/extraction_candidates.json. Candidates are only original-text fragments, not questions. After reading candidates, combine the target role and resume,each candidatewrite oneExtractionDecision(candidate_id, accepted, canonical_text, topic, role_tags)and usesave_extraction_decisionsto write intocorpus_cache/extraction_decisions.json:accepted=trueonly for real, relevant, and semantically clear interview questions;canonical_textuse one sentence, as short as possible, in Chinese to summarize the question's intent, removing phrasing such as "please introduce/how do you view/the interviewer asked", but technical entities, business objects, and key conditions must be preserved. For example, "how to evaluate recruitment channels" and "how should recruitment channels be evaluated" are both written as "evaluate recruitment channels"; "Redis cache penetration" and "Redis cache avalanche" cannot be merged.- Do not provide question text, URL, date, or original-text evidence. They are generated by Python from the candidates in a fixed way; rejected candidates must also be written
accepted=false, and must not be silently skipped.
5a. Generate a reproducible run package. After the decisions are written, execute:
python -m scripts.corpus.runner \
--raw-posts corpus_cache/raw_posts.json \
--decisions corpus_cache/extraction_decisions.json \
--output corpus_cache/runs/<本次标识> \
--today YYYY-MM-DD
--output must be anot-yet-existingdirectory. The runner will regenerate candidates from raw_posts , then complete materialize_questions()、prepare_questions() and deduplication/ranking; it writes the input copy, intermediate artifacts, ranked questions, rejected items, diagnostics, and the SHA-256 manifest into that directory. The manifest also records the todayactually used for freshness judgment; formal reruns should explicitly pass the same date. The final question text still comes from the candidates' display_text, and the original-text evidence comes from the candidates' excerpt, and must not be manually replaced.
5b. Read data gaps. First check <输出目录>/diagnostics.json: it contains decision processing results (including candidates explicitly not retained or missing decisions), candidate/evidence issues, expired-source counts, and rejected items. It does not execute search, crawling, resume reading, semantic decision, or final study-guide copywriting; these remain explicit steps before and after. The formal study package is only allowed to read <输出目录>/ranked_questions.json; it must not fall back to materialized_questions.json, old caches, or manually assembled questions.
-
Deduplication & ranking. Inside the runner,
prepare_questions()has completed evidence and freshness validation, deduplication and ranking bycanonical_text. Ranking prioritizesthe number of independent source URLs and freshness, the occurrence count is only used to break ties when scores are equal. Repeated occurrences of the same URL cannot masquerade as high frequency from multiple sources. Only questions supported by at least 2 independent sources verified through evidence are called "high frequency"; the rest may be kept as "related questions," but must be clearly identified as single-source. -
Project-anchored reasoning. For each high-frequency question, check whether it can be attached to some project/skill in the resume. If it can be attached, construct
FollowUpChain(seed question → personalized follow-up,is_grounded=true). Every follow-upmustbe traceable to (some resume project/skill) + (some actually crawled question); if it cannot be traced, setis_grounded=false, and keep it as an ordinary standard interview question.Do not fabricate follow-ups out of thin air. -
Study package. Write Markdown strictly according to the fixed template below, save it to
corpus_cache/prep_package.mdand show it to the user. Do not freely improvise or change the major structure; if some part has insufficient data, clearly write "insufficient data/not covered"; do not make things up.
Fixed study package template
# {目标岗位}岗位备考包 — {候选人姓名或"候选人"}
生成日期:{YYYY-MM-DD}
目标岗位:{用户给定岗位 + 从真实数据收割到的岗位别名}
## 1. 你的候选人定位
用 1 段话定义候选人的面试包装方向。格式:
> {一句候选人定位}
简历里最强的三个证据:
1. **{项目/经历 1}**
- {证据点 1}
- {证据点 2}
- {适合回答什么面试问题}
2. **{项目/经历 2}**
- ...
3. **{项目/经历 3}**
- ...
## 2. 岗位 Gap 分析
| 维度 | 当前简历表现 | 面试风险 | 准备建议 |
|---|---|---|---|
| {能力维度} | {基于简历的证据} | {真实面经会追问的风险} | {可执行补强建议} |
至少覆盖:技术理解、产品基本功、业务 sense、数据/实验、落地可信度。非 AI 岗位时把维度换成该岗位的核心能力。
## 3. 真实数据来源概况
本次召回:
- {来源 1}:{数量/状态/保存路径}
- {来源 2}:{数量/状态/保存路径}
数据缺口:
- {降级源、OCR/反爬/时间戳缺口等}
## 4. 高频题 Top {N}
### {序号}. {题目}
来源:
- {source title 或 source type}:`{url}`
- 原文证据:`{QuestionEvidence.excerpt}`
- ...
回答要点/回答框架:
- {要点 1}
- {要点 2}
- {要点 3}
可挂简历锚点:
> {把这题连接到候选人某个项目/技能的一段话}
如果适合表格解释,可以用小表格;否则用 bullet。每题必须至少有 1 个真实来源。没有来源的题不能进 Top 高频题。
## 5. 个性化项目追问链
### 链 {序号}:{主题} → {简历项目/经历}
种子题:{来自第 4 节的高频题}
追问:
1. {追问 1}
2. {追问 2}
3. {追问 3}
4. {追问 4}
5. {追问 5}
准备重点:
- {如何准备真实例子/图/指标}
- {面试时要强调的产品视角}
每条追问必须同时能追溯到"真实面经题目"和"简历项目/技能";追溯不上就不要写成个性化追问。
## 6. 你的 60-90 秒自我介绍草稿
写 2-3 段中文口语稿。必须包含:
- 背景
- 目标岗位动机
- 2-3 个最强项目证据
- 候选人的差异化定位
## 7. 一周冲刺计划
### Day 1:{主题}
- {行动项}
- {行动项}
...
### Day 7:{主题}
- {行动项}
- {行动项}
## 8. 建议你立刻补强的简历表述
### {项目/经历 1}
补:
> {可直接放进简历/面试话术的改写}
### {项目/经历 2}
补:
> ...
## 9. 来源列表
按来源类型分组列出代表来源:
小红书:
- `{url}` — {一句说明}
牛客/网页/GitHub:
- `{url}` — {一句说明}
## 10. 面试前速查清单
- 60 秒自我介绍
- {3 个最能证明岗位匹配度的项目/经历证据}
- {3 个真实 trade-off / 失败 / 返工案例}
- {5 个目标岗位必会高频题;例如 AI 产品岗可写 RAG、Agent、实验、指标、产品设计,其他岗位按真实面经替换}
- {4 个高质量反问}
- {可以展示的作品、流程图、数据、文档或代码证据}
Template constraints:
- The full text is in Chinese, but technical terms may remain in English.
- The questions in Section 4 may only come from this run package's
<输出目录>/ranked_questions.json; each of itsQuestionEvidence.excerptmust be traceable to this run package'sraw_posts.json. Xiaohongshu caption/tag are only used aslocator_textassistance and cannot generate questions on their own. - Do not directly input arbitrary
Question、RawPost.content_text/raw_textinto the template or manually callprepare_questions()results to bypass the run package. Questions with only a URL and no original text snippet cannot enter the high-frequency section. - High-frequency questions are preferentially sorted by
dedupe_and_rankresults; manual adjustments may only be made due to job relevance or resume match, and the basis must be explained. Questions called "high frequency" must have at least 2 independent source URLs; if insufficient, rewrite as "Related Questions Top N". - Source URLs must actually exist in
source_refsor RawPost.url; do not write untraceable sources such as "comprehensive materials". - Self-introduction and resume wording may be optimized for expression, but the facts must come from the resume or real interview experiences.
- Section 10 must be dynamically generated according to the user's target job and must not be hardcoded as AI product questions; AI product is only an example.
Constraints
- All user-facing output must be in Chinese(study package, questions, follow-ups, analysis) — the interview experience sources are in Chinese.
- Current sources: Nowcoder + Xiaohongshu (primary, with timestamps) + GitHub (supplementary).
- Xiaohongshu is collected and exported via MediaCrawler (the user runs it offline once in advance; see the process in
docs/setup/mediacrawler.md); OCR uses a hybrid strategy (coarse OCR + visual fallback); MediaCrawler is for personal, non-commercial use only. - Timeliness is a hard requirement: sources with explicit dates are kept only for the past two years; undated supplementary sources must be disclosed separately and sorted by the lowest weight.
- Third-party crawlers used in later versions (such as MediaCrawler) are for personal, non-commercial use only.
- Traceability takes priority over fluency: never fabricate questions or follow-ups that cannot be traced to real data.
- If the connector returns
degradedand the message containsselector, it means the source site's HTML has changed selectors; go to the correspondingscripts/connectors/<name>.pytop comment to see the current assumptions, verify the real HTML, then update the selectors and add fixtures.