SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsdata-analysis

data-analysis

Data & analytics

View repository

Updated 10-15 20:05

About this skill

Data Analysis Skill

Analysis Objectives

The core objectives of a data analysis task are:

  1. Understand the data: figure out what the data is, where it comes from, and what business meaning it represents, rather than just reading the file
  2. Assess data quality: identify issues such as missing values, duplicates, anomalies, and inconsistent formats, and judge the credibility of the data
  3. Discover patterns and insights: through statistics and exploration, find meaningful patterns, trends, anomalies, and associations in the data
  4. Formulate a processing workflow: based on actual data issues, design a reproducible and well-founded cleaning and transformation plan
  5. Summarize results: output conclusions in clear language with business value, not a pile of statistical figures

Processing Constraints

During the analysis process, always comply with the following constraints:

  • Do not modify the original data without authorization: any cleaning or transformation operation must first be communicated to the user, with the reason explained, before execution
  • Do not over-assume the meaning of the data: when column names are unclear, confirm with the user instead of guessing
  • Do not ignore data quality issues: anomalies, missing values, and inconsistencies must be clearly flagged, not bypassed
  • Do not force the application of a fixed workflow: data formats vary greatly, so the analysis path should be judged flexibly according to the actual situation
  • Do not conduct analysis detached from the user's goals: always decide the depth and direction of the analysis around "what problem does the user want to solve"

Best Practices

Understand first, then act

After receiving the data, first clarify the background: what is the business scenario of the data? What is the user's analysis purpose? This determines which columns are important, what counts as an "anomaly", and how to handle missing values.

Explain results in business language

Do not just output statistical figures; explain their meaning. "The mean is 3200" is not as good as "The average order amount is about 3,200 yuan, but the median is only 1,800 yuan, indicating that a small number of high-value orders pulled up the mean."

Handle data quality issues by severity

  • Blocking issues(such as all key columns being empty or a large number of duplicate primary keys): stop first and inform the user, then continue after confirmation
  • Issues requiring a decision(such as a high missing rate or anomalous values with unclear meaning): list the pros and cons of processing options and let the user choose
  • Minor issues(such as a small number of inconsistent formats or whitespace characters): can be handled directly, but must be recorded in the report

The processing workflow must be explainable

Every step must be able to answer "why do it this way", for example:

  • "Fill with the median because the column distribution is right-skewed and the mean is heavily affected by extreme values"
  • "Delete this column because the missing rate is 78%, making it impossible to use effectively"

Conclusions must be supported by data

Every insight should be followed by concrete numbers as evidence, avoiding subjective judgment.

Environment and Tools

  • File paths: use <workspace>/uploads/for the upload directory, and use <workspace>/outputs/
  • Common encoding issues with Chinese data: try utf-8first; if that fails, try gbk / gb18030
  • Chinese display in charts: set matplotlib.rcParams['font.sans-serif'] = ['SimHei', 'DejaVu Sans']
  • When outputting CSV for opening in Excel, use encoding='utf-8-sig' to prevent garbled text

Analysis Results Summary Format

Regardless of the analysis path, the final summary report should include:

## 数据分析报告

### 数据概况
（数据规模、来源、时间范围、核心字段说明）

### 数据质量
（发现的问题、严重程度、已做处理或待确认事项）

### 核心发现
（3-5 条最重要的洞察，每条附数据支撑）

### 处理流程说明
（对数据做了哪些变换，为什么这样做）

### 后续建议
（数据改进建议、可深入分析的方向）

Examples

The following examples show how to apply the above goals and constraints when analyzing different types of data.

Example A: E-commerce order data analysis

Scenario: The user uploads an order CSV with columns including order ID, order time, product category, amount, user city, and payment status, and wants to understand the "overall sales situation."

Analysis approach：

  • First confirm the time range; this is the basic context for sales analysis
  • Check the amount column for negative values (refunds? data entry errors?) and zero values (which require business explanation)
  • The distribution of payment status determines the definition of "valid orders"
  • Aggregate by time to see trends, and by category and city to see composition

Example of handling data quality issues：

Found that the "amount" column has 12 rows with negative values. In sales data, negative values usually represent refund records. Recommendation: if analyzing sales revenue, filter out negative values; if analyzing net income, keep them. Please confirm the analysis goal before proceeding.

Example of key findings：

  • March sales increased 34% month over month, mainly driven by the "home appliances" category (contributing 61% of the增量)
  • Beijing and Shanghai together account for 47% of all orders, but their average order value is 15% lower than the average of second-tier cities
  • The payment failure rate is 8.3%, higher than the industry reference value (usually 2-5%); it is recommended to investigate the payment pipeline

Example B: Poorly structured survey questionnaire data

Scenario: The user uploads an Excel file with multiple sheets, inconsistent column names (some called "Q1", some called "Question 1"), some merged cells, and empty rows and columns.

First step: Do not rush into analysis; first explain the data structure issues to the user:

This file contains 3 sheets, and the structure has the following issues that need to be confirmed first:

  1. The column names in Sheet1 and Sheet2 are inconsistent. Are they different batches of the same questionnaire, or different questionnaires?
  2. Rows 5-8 are empty rows. Can they be deleted?
  3. The "Q3_other" column is 92% empty. Is this because the response rate for the open-ended question is low, or was there a problem during export?

After confirmation, I will formulate a cleaning plan.

Reflecting constraints: Here, do not assume the meaning of column names, do not merge sheets without authorization, and first expose the issues and wait for user confirmation.

Example C: The user requests "clean the data and output the processed file"

Example of processing workflow documentation(provided together with the output file):

数据处理流程记录
原始文件：sales_2024.csv（8,412 行 × 15 列）
处理后文件：sales_2024_cleaned.csv（8,203 行 × 13 列）

变更说明：
1. 删除重复行：移除 89 行完全重复记录
2. 删除"备注2"列：缺失率 96%，无有效信息
3. 删除"内部编码"列：用户确认该列不参与分析
4. 金额列格式统一：将"¥1,200.00"格式转为数值 1200.0（共 203 行受影响）
5. 日期列标准化：统一转为 YYYY-MM-DD 格式（原始混有 MM/DD/YYYY 和中文日期两种格式）
6. 缺失值处理：
   - "城市"列 34 行缺失 → 填入"未知"（用户确认）
   - "金额"列 86 行缺失 → 保留为空（用户确认这批订单为异常记录，不填充）