xgb-modeling

XGBoost Modeling (portable)

Performs binary classification modeling based on XGBoost, supporting automatic feature selection, multi-scheme comparison, and stability analysis. Can run in any Python environment, with no platform coupling. For hyperparameter tuning please use xgb-tuning。

Core Workflow: data loading → feature selection (four major operators) → multi-scheme comparison → optimal scheme training → three-stage evaluation → stability analysis → report generation

Parameters

The general parameter spec is defined in _vendor/xgb_cli.py（domain=modeling）。

Parameter	Required	Default Value	Description
--data_path / -d	✅	-	Data file path (parquet/csv)
--target / -t	✅	-	Target variable column name (0/1 binary classification)
--time_col		busi_dt	Time column name
--train_filter		Auto split	Training set filter condition (pandas query)
--val_filter		val_ratio split out	Validation set filter condition
--oot_filter		Split by time	OOT cross-time test set condition
--oot_ratio		0.20	When --oot_filter is not passed, the proportion by which OOT is split by time
--val_ratio		0.25	When --val_filter is not passed, the proportion by which val is split from train_full
--random_seed		42	Random seed
--exclude_cols		-	Excluded columns, comma-separated
--features		-	Specify a single feature set, comma-separated
--feature_sets		-	JSON multi-scheme;For complex nesting, prefer putting it in --config 's feature_sets field
--feature_scheme		-	Specify a single automatic selection scheme:full/decorr/high_iv/stable; if not passed or passed empty, go through --auto_select full comparison
--auto_select		true	Automatic feature selection (full comparison of the four major operators; when --feature_scheme is specified, this parameter is ignored)
--baseline_filter		Same as train	PSI baseline condition
--comparison_filter		Same as oot	PSI comparison condition
--sample_strategy		auto_weight	auto_weight / undersample / none
--model_name		Auto-generated	Model name (without extension)
--report_output		xgb_modeling_report	Report base name
--output_dir		./outputs/<ts>	portable-exclusive: artifact output directory
--config		-	JSON config file path (command line > config > default)

For complex nested parameters (such as feature_sets), do not assemble escaped JSON on the command line, instead use the --config config.json structured channel.

---

Execution Method

python scripts/modeling.py \
  --data_path ./examples/toy.parquet \
  --target y_label \
  --time_col busi_dt \
  --output_dir ./outputs/run1

Complex feature_sets is passed in via --config :

python scripts/modeling.py \
  --data_path ./examples/toy.parquet \
  --target y_label \
  --config ./config.json \
  --output_dir ./outputs/run1

After execution ends, the following are generated under <output_dir>/ :

• xgb_modeling_report.md — modeling report (data overview, scheme comparison, optimal scheme, stability analysis…)
• models/<model_name>.json — XGBoost model file
• models/<model_name>_meta.json — model metadata (feature list, etc.)
• models/<model_name>_card.json — ModelCard
• result.json — structured artifact manifest (see PROTOCOL.md)

The last line of stdout prints the absolute path of result.json ; the Agent can read this file to obtain all artifact paths and metrics.

Artifact Example (result.json)

{
  "skill": "xgb-modeling",
  "status": "success",
  "files": [
    {"path": ".../xgb_modeling_report.md", "role": "report"},
    {"path": ".../models/xgb_model_20260512_143000.json", "role": "model",
     "meta": {"feature_count": 8, "scheme_name": "去共线性", "model_type": "xgboost"}},
    {"path": ".../models/xgb_model_20260512_143000_meta.json", "role": "meta"}
  ],
  "metrics": {
    "oot_auc": 0.7823, "oot_ks": 0.4215,
    "auc_gap": 0.0214, "ks_gap": 0.0312,
    "n_features": 8, "overall_score_psi": 0.0423
  },
  "summary": "xgb-modeling 完成：最优方案「去共线性」，OOT AUC=0.7823，OOT KS=0.4215，特征数=8，Gap=0.0214"
}

Cross-skill Chaining

Downstream (such as model-explanation) via --model_path Path in the upstream result.json passed in: role=model File paths:

# 从上游 result.json 提取模型路径
MODEL_PATH=$(jq -r '.files[] | select(.role=="model") | .path' ./outputs/run1/result.json)

python ../model-explanation/scripts/explain.py --model_path "$MODEL_PATH" --data_path ./examples/toy.parquet

---

Common Scenarios

Scenario 1: Automatic feature selection modeling

Without specifying features, automatically run the four major operators to generate four schemes for comparison:

python scripts/modeling.py --data_path ./examples/toy.parquet --target y_label

Scenario 2: Modeling with specified features

Use the specified feature list:

python scripts/modeling.py --data_path ./examples/toy.parquet --target y_label --features "feat1,feat2,feat3"

Scenario 2-b: Specify a single automatic selection scheme

Run only one selection scheme (e.g., "remove collinearity"):

python scripts/modeling.py --data_path ./examples/toy.parquet --target y_label --feature_scheme decorr

Available values:full(full features into model) / decorr(remove collinearity) / high_iv(high predictive power) / stable(stability first)

Scenario 3: Multi-scheme comparison

Pass in multiple feature schemes (Prefer --config, avoid escaping when assembling JSON on the command line):

{
  "feature_sets": {
    "方案A": ["f1", "f2"],
    "方案B": ["f1", "f3"]
  }
}

python scripts/modeling.py --data_path ./examples/toy.parquet --target y_label --config ./config.json

Scenario 4: Save the model

After training is complete, save the optimal model for subsequent use by model-explanation:

python scripts/modeling.py --data_path ./examples/toy.parquet --target y_label --model_name my_best_model

The model is saved automatically, output:

• Model file:<output_dir>/models/my_best_model.json
• Metadata file:<output_dir>/models/my_best_model_meta.json(including feature list, parameters, etc.)

---

Report Structure

Section	Content
Executive Summary	4 lines of business conclusions: optimal scheme, Gap status, PSI status, top feature risks
1. Data Overview	Sample size, positive sample rate (Train/Test/OOT)
2. Feature Scheme Comparison	Horizontal comparison of AUC/KS/Gap across schemes
3. Optimal Scheme Details	AUC/KS/Gini metrics, optimal binning of model score IV, Lift table, feature importance, BCR @ Top5/10/20/30%, Brier Score + calibration curve
4. Stability Analysis	Monthly AUC/KS (including 95% Bootstrap confidence interval), Mann-Kendall trend test, score PSI
5. Final Model Summary	Feature list, comprehensive performance

---

Comparison with Other Modeling Skills

Dimension	xgb-modeling	lr-modeling	dnn-modeling
Algorithm	XGBoost	Logistic Regression + WoE	MLP (PyTorch)
Feature Processing	Raw values directly input	WoE binning encoding	StandardScaler
Nonlinear Capability	Strong (tree structure)	Weak (introduced only through binning)	Strong (multi-layer activation)
Interpretability	Medium (requires SHAP)	Strong (coefficient × WoE)	Weak
Training Speed	Fast	Very fast	Slow
Applicable Data Volume	Any	Any	>10k
Evaluation System	AUC/KS/BCR/PSI	AUC/KS/BCR/PSI	AUC/KS/BCR/PSI

---

Upstream and Downstream Relationships

• Predecessor：data-profiling → feature-analysis(feature selection)
• Successor：xgb-tuning(hyperparameter tuning optimization),model-explanation(SHAP explanation)
• Parallel: can be compared horizontally with lr-modeling / dnn-modeling (same data, different algorithms)

---

Dependencies

See requirements.txt(Core:xgboost, scikit-learn, optbinning）。

---

Notes

1. Target Variable: Must be 0/1 binary classification
2. Time Splitting: It is recommended to split by time to ensure OOT is future data
3. Automatic feature selection: Data must simultaneously satisfy the IV, PSI, and missing rate conditions
4. Sample Strategy: For highly imbalanced scenarios (positive sample rate < 2%), it is recommended to use --sample_strategy undersample; for general scenarios, use the default auto_weight
5. No hyperparameter tuning provided: If hyperparameter tuning is needed, switch to xgb-tuning (constraint search based on Optuna TPE + diagnostics-driven)
6. Model Saving: Each training automatically saves the optimal model to <output_dir>/models/, no need to pass --save_model; you can use --model_name to customize the file name
7. Stability Analysis: The baseline month is taken from the training set time period; Bootstrap CI is only calculated when the sample size is >= 100
8. BCR/Calibration Curve: BCR (Bad Capture Rate) reflects how many bad customers can be captured by rejecting the top K% of the population; the calibration curve reflects the credibility of the model's probability output