data-profiling

Data Insight Report (portable)

Based on scripts/profiler.py main script, performs a profile scan on the dataset and generates a data overview report.

Parameter Description

Parameter	Required	Default Value	Description
`--data_path`	✅	-	Data file path (parquet/csv)
`--output`		`data_profiling_report.md`	Report output path
`--output_dir`		`./outputs/<ts>`	Artifact output directory
`--output_name`		`data_profiling_report`	Report base name (without extension)
`--config`		-	JSON configuration file path

Execution Method

python scripts/profiler.py \
  --data_path ./examples/toy.parquet \
  --output_dir ./outputs/profile_run

After execution ends:

Artifact directory <output_dir>/ generates:
- report.md — Data overview report
- result.json — Structured artifact list (see PROTOCOL.md)
The last line of stdout prints the absolute path of result.json , and the Agent can read this file

Artifact Example

{
  "skill": "data-profiling",
  "status": "success",
  "files": [
    {"path": ".../report.md", "role": "report"}
  ],
  "metrics": {"n_rows": 10000, "n_cols": 28, "n_missing_cols": 5},
  "summary": "中等规模数据集（10,000行），28 个字段，数值型为主，发现 2 个高缺失字段"
}

Report Output Structure

Section	Content
1. Data Overview	File name, format, size, row and column counts, field type distribution pie chart
2. Field Details List	Each field's type, missing rate, unique value count, example values
3. Numerical Feature Analysis	Descriptive statistics (mean/std/quantiles/skewness/kurtosis) + histogram distribution chart
4. Categorical Feature Analysis	Unique value count, Top value proportion, concentration + bar chart
5. Missing Value Analysis	Missing rate ranking table + bar chart
6. Data Quality	Duplicate rows, empty columns, constant columns, high-missing columns
7. Sample Preview	Display the first 5 rows of data

Adaptive Display

When the number of numerical/categorical features exceeds 20, the statistical tables and charts only display the first 20, to avoid the report becoming too long.

Relationship with Other Skills

Skill	Purpose Difference
data-profiling	Quickly understand the data profile, without deep analysis
feature-analysis	Deep feature analysis (IV, PSI, correlation, etc.), requires a target variable
xgb-modeling	Full modeling workflow, requires a target variable

Suggested Process：

First use data-profiling to understand the basic situation of the data
Based on the insight results, decide whether feature-analysis or xgb-modeling

Notes

No target variable required: This Skill does not require a target variable, it is purely data description
Fast and lightweight: Fast execution speed, suitable for quick scanning of large datasets
Automatic type inference: Identifies numerical, categorical, boolean, datetime, and text fields