cjournal-analyzer

Comprehensive Analysis Tool for C-Journal Papers

Phase 1: Determine the journal and CNKI code

Extract the journal name from user input
Query references/journal_codes.md to obtain the CNKI code (e.g., "Management World" → GLSJ）
If not included, use WebSearch to search site:navi.cnki.net/knavi/journals "{期刊名}" Extract the code from the URL
Confirm the journal with the user before continuing

Phase 2: Browser data collection

Use Chrome DevTools MCP tools to collect data from CNKI.

Step 2.1: Open the journal page

navigate_page → https://navi.cnki.net/knavi/journals/{CODE}/detail

CAPTCHA handling: If the page shows "Please complete the security verification" or "Drag the puzzle below", immediately prompt the user:

"CNKI requires security verification. Please complete the slider verification in the browser, and let me know when done."

After the user confirms, use navigate_page to reload the page.

Step 2.2: Extract basic journal information

() => {
  const title = document.querySelector('h3')?.textContent?.trim() || '';
  const info = {};
  document.querySelectorAll('.detailInfo p, .s-info p').forEach(p => {
    const text = p.textContent;
    if (text.includes('主办单位')) info.sponsor = text.split('：')[1]?.trim();
    if (text.includes('ISSN')) info.issn = text.split('：')[1]?.trim();
    if (text.includes('CN')) info.cn = text.split('：')[1]?.trim();
    if (text.includes('出版周期')) info.frequency = text.split('：')[1]?.trim();
    if (text.includes('复合影响因子')) info.cif = text.split('：')[1]?.trim();
    if (text.includes('综合影响因子')) info.aif = text.split('：')[1]?.trim();
  });
  return { title, ...info };
}

Basic information can also be read directly from the snapshot (StaticText nodes).

Step 2.3: Click the "Papers" tab and extract years and issues

Click the link corresponding to the "Papers" uid, wait for loading, then extract:

() => {
  const results = [];
  document.querySelectorAll('dl[id$="_Year_Issue"]').forEach(dl => {
    const year = dl.querySelector('dt em')?.textContent?.trim();
    if (!year) return;
    const issues = [];
    dl.querySelectorAll('dd a').forEach(a => {
      issues.push({ id: a.id, issue: a.textContent.trim(), value: a.getAttribute('value') });
    });
    results.push({ year: parseInt(year), issues });
  });
  return results;
}

Filter data from the most recent 5 years (current year - 4 to current year). If CNKI paginates years (each page shows only some years), you need to turn pages to load more.

Step 2.4: Collect article lists issue by issue

For each issue, click the corresponding issue link (via click uid or evaluate_script simulated click), wait for the article list to load (wait_for wait for the title to appear or wait 1-2 seconds), then extract:

() => {
  const articles = [];
  document.querySelectorAll('#CatalogList dd.row').forEach(dd => {
    const titleEl = dd.querySelector('span.name a');
    const authorEl = dd.querySelector('span.author');
    const pageEl = dd.querySelector('span.company');
    const sectionEl = dd.closest('div')?.querySelector('dt.tit');
    if (titleEl) {
      articles.push({
        title: titleEl.textContent.trim(),
        url: titleEl.href,
        authors: authorEl?.getAttribute('title')?.replace(/;$/,'') || '',
        pages: pageEl?.getAttribute('title') || '',
        section: sectionEl?.textContent?.trim() || ''
      });
    }
  });
  return articles;
}

Key：

Keep a 1-2 second interval between collections for each issue to avoid triggering anti-scraping
Continuously report progress to the user (e.g., "Collecting 2024 Issue 6, completed 32/60 issues...")
All data is temporarily stored in a JSON array

Step 2.5: Abstract collection (sampling strategy)

Collecting all abstracts takes an extremely long time, so use sampling: select 2 issues per year (e.g., Issue 1 and Issue 7), and visit the abstract pages of the first 3 articles in each issue.

After visiting the article detail page, extract the abstract:

() => {
  const abs = document.querySelector('#ChDivSummary, .abstract-text, [name="abstracts"]');
  const kw = document.querySelector('#ChDivKeyWord, .keywords');
  return {
    abstract: abs?.textContent?.trim() || '',
    keywords: kw?.textContent?.replace('关键词：','').trim() || ''
  };
}

If the browser method is blocked, use WebSearch to search "{文章标题}" site:cnki.net to supplement abstracts and keywords.

Phase 3: Data analysis

After collection is complete, save all data as JSON, then run scripts/analyze_journal.py for analysis.

The script depends on:pip3 install jieba wordcloud python-docx matplotlib numpy

The script receives the JSON data file path and outputs analysis results and visualization charts:

Publication volume trend: line chart of annual publication counts
Top 30 high-frequency keywords: jieba segmentation → stopword removal → word frequency statistics → bar chart + word cloud
Topic clustering: coarse-grained topic categorization based on co-occurrence of high-frequency words
Column analysis: pie chart of publication share by column and trend changes
Top 20 core authors: bar chart of publication frequency
Research method identification: Match method keywords from the title (empirical/case study/experiment/model/simulation/survey/interview/bibliometrics/meta-analysis/regression/panel data/DID/RDD/PSM/machine learning/deep learning/SEM/grounded theory, etc.)
Hotspot Evolution: Compare keywords from the previous 3 years vs the recent 2 years to identify emerging/declining topics
Research Gaps and Submission Suggestions: Provide a comprehensive summary based on the above analysis

Phase 4: Generate Word Report

Use python-docx to generate a formatted report, with the structure:

封面：《{期刊名}》近五年（{起始年}-{结束年}）发文分析报告

一、期刊概况
二、发文量与趋势分析（含图表）
三、选题热点分析（含词云图、高频词柱状图）
四、热点演变与新兴主题
五、核心作者群分析
六、研究方法偏好分析
七、栏目主题分析
八、研究空白与投稿建议
附录：完整文章目录（按年份-期次排列）

Save the report to ~/Downloads/{期刊名}_近五年发文分析报告.docx。

Notes

CNKI has anti-scraping mechanisms; the interval between each request must be ≥1 second
When a CAPTCHA appears, the user must be asked to complete it manually
Install dependencies:pip3 install jieba wordcloud python-docx matplotlib numpy
For journal codes, check first references/journal_codes.md
Keep progress announcements throughout the collection process