data-analysis

Data analysis

After the user uploads a data file, follow these guidelines to read it, reason about it, and produce a report.

Data loading guidelines

Use pandas to read the user's uploaded files, adapting to the situation:

• Excel files: first check the names of all worksheets. If there is only one, read it directly; if there are several, list them for the user to choose.
• CSV files: detect the file encoding with charset-normalizer (or chardet), then read using the detected encoding to avoid silent text corruption caused by trying encodings one by one.
• Header detection: check whether the first row can reasonably serve as column names. If it contains only numbers, the header may be missing.
• Immediately after loading, print the row count, column names, and first 3 rows to confirm the data was read correctly.

These are guidelines, not a fixed script. Every dataset differs and requires judgment based on its actual circumstances.

Analysis methodology

Stage one: understand the structure

Before computing any statistics, answer:

• What does one row represent? (An advertising placement? A user? A daily summary?)
• Which columns aredimensions(category columns used for grouping, such as "segment" or "channel")?
• Which columns aremetrics(numeric columns used for calculation, such as "GMV" or "ROI")?
• Is there a time column? What time range does the data cover?

Stage two: statistical overview

For each metric column:

• Calculate both the mean and median. A difference greater than 20% indicates a skewed distribution; in this case, the median better represents a "typical value."
• Calculate p25, p75, and p95 (the values at the 25th, 75th, and 95th percentiles) to understand the distribution's range.
• Count zero values and missing values.

Group by dimension columns and compare metrics across groups.

Stage three: uncover findings

• Anomaly detection: Find records more than 2 standard deviations from the within-group mean.
• Correlation: Calculate correlation coefficients between metrics and flag strong correlations (|r|>0.7).
• Trends: If there is a time column, check how metrics change over time.

Analysis philosophy

Data analysis aims to answer "so what" (what it means and what business value it has), rather than merely present numbers:

• Comparison creates meaning: "A 7.7% return rate" is a lifeless number; "the apparel return rate is 4 times the cosmetics return rate" has impact.
• Answer "so what": Follow every finding with the question "What does this mean?"
• The heading is the conclusion: Write "Food-segment ROI keeps falling and needs an alert," rather than "Food-segment analysis," so readers understand what happened from the heading.
• Outliers are a gold mine: Data points that deviate from the mean are often the most valuable.
• Actively uncover hidden issues: After the analysis, proactively ask "What other trends or risks might the user have missed that the data already suggests?"
• Beware of false precision: "About 5%" is more honest than "4.73%."

Output structure (for the brief conversational report)

Before generating HTML, briefly report the analysis results in the conversation using this structure:

One-sentence summary

(The single most important finding in the dataset, in no more than two sentences)

Data overview

(Row count, column count, time range, and distributions of the main dimensions)

Key findings (3–5, ordered by importance)

(Each includes the finding + supporting data + so what)

Points to watch

(Outliers and data quality issues)

Recommendations

(1–3 actionable recommendations based on the findings)

Report generation workflow

After completing the analysis, generate a standalone HTML report file:

1. First read references/design-brief.md to understand the design direction.
2. Then read references/report-template.html to understand the report's base structure and available components.
3. Select suitable components from the template based on the findings and generate a customized HTML report.
4. Use Chart.js (loaded from a CDN) for charts, and embed the data in the HTML as JS variables (const DATA = {...}); ensure the report is a standalone single file that can be opened offline.

Output conventions:

• Save the file in the same directory as the input data.
• The filename is <原文件名>-report.html
• Tell the user the report's saved path in the conversation.

Chart selection guide

Include at least one chart for every key finding. Choose the chart type based on the relationship in the data:

• Category comparison → bar chart
• Time trend → line chart
• Composition → doughnut chart (no more than 6 categories)
• Variable relationships → scatter plot
• Distribution analysis → histogram

Design direction

When generating HTML, refer to the design brief in references/design-brief.md and use its guidance to make your own decisions about layout, colors, typography, and other design details.

Core requirements:

• Style: editorial magazine layout; avoid an admin-dashboard style.
• References: The Economist's chart pages, McKinsey research reports, and Bloomberg data visualizations.
• Chart colors must match the report's overall style; do not use Chart.js's default colors.
• Make key numbers large and prominent, supporting information small and quiet, and leave ample spacing.
• Choose specific color values, fonts, spacing, and other parameters yourself.

Note

• Use Chinese for all output.
• Keep numbers to two decimal places.