eda-sports

EDA for Sports Data

Outcome

Understand the data well enough that the next decision is explicit. Sports EDA is not one call to describe(): it is a structured review of grain, keys, time coverage, entities, missingness, targets, schedule structure, outliers, structural breaks, and fields unavailable at prediction time T.

Finish with a written go, repair, or stop decision. A clean EDA report does not certify that engineered features are time-safe.

When to Use This Skill

Use when:

• a sports table is new, freshly loaded, or not yet trusted;
• grain, keys, coverage, missingness, or base rates need an honest audit;
• before feature engineering or first model fit;
• a weird model result might just be bad data structure.

Do not use this skill as a substitute for:

• formal leakage verdicts → leakage-audit;
• feature construction → feature-rules / time-series-sports;
• source selection → data-sources.

Need	Go instead
Choose a public source	data-sources
Build legal features	feature-rules
Prove time-safety	leakage-audit
Charts only	sports-visualization

Required context

Before calculating summaries, record the question, target, prediction time T, claimed row grain and key, source/retrieval time, requested period, completion status, and populations included. If grain or T is ambiguous, resolve it first.

Use the EDA checklist as the full run sheet. Read the grain guide when row meaning, keys, or aggregation are in doubt. Use the red-flags guide when a count, base rate, or model result is odd.

Workflow

1. Define one row: game, team-game, player-game, play, possession, pitch, or event.
2. Count rows/columns; test natural-key uniqueness and rows per contest.
3. List seasons, weeks/rounds, dates, gaps, partial periods, and active seasons.
4. Count teams/players by season and inspect ID churn and failed joins.
5. Measure missingness overall and by season, source, role, and completion state.
6. Inspect targets, base rates, impossible values, ties, overtime, and zero inflation.
7. Slice meaningful predeclared groups: home/away, phase, role, venue, and era.
8. Find rule, schedule, provider, tracking, definition, and population breaks.
9. Flag fields unavailable at T; reserve the formal verdict for leakage-audit.
10. Write findings, repairs, limitations, and the decision.

If grain is wrong or keys are unexplained duplicates, stop before modeling.

Grain and key diagnostics

Grain	One row	Typical key	Main trap
Game	one contest	game_id	mixing team perspectives
Team-game	one team in one contest	game_id, team	treating doubled rows as games
Player-game	one player in one contest	game_id, player_id	ignoring DNP/inactive policy
Play/event	one event	game_id, event_id	using post-event fields at pre-event T
Pitch	one pitch	game_pk, pitch_no	unstable event ordering

key = ["game_id", "team"]
dupes = panel.duplicated(key, keep=False)
print("duplicate key rows:", int(dupes.sum()))
print(panel.loc[dupes].sort_values(key).head(20))
print(panel.groupby("game_id", dropna=False).size().value_counts().sort_index())

Do not call drop_duplicates() until the cause is known. Repeated pulls, provider revisions, multiple competitions, and mixed grains need different repairs. A normal team-game panel usually has two rows per game.

Coverage and entity balance

coverage = (
    panel.groupby(["season", "week"], dropna=False)
    .agg(rows=("game_id", "size"), games=("game_id", "nunique"),
         teams=("team", "nunique"))
    .reset_index()
)
print(coverage.to_string(index=False))
print(panel.groupby(["season", "team"]).size().unstack(0).fillna(0))

Investigate empty periods, low game counts, abrupt entity-count changes, and the tail of an active season. Imbalance can be real—byes, playoffs, injuries, promotion/relegation—but can also reveal missing loads. Use stable IDs and a documented crosswalk for relocations and renames.

The portable coverage helper expects season, week, game_id, and team. Map alternate schemas explicitly:

python /path/to/eda-sports/scripts/coverage_table.py --input games.csv
python /path/to/eda-sports/scripts/coverage_table.py \
  --input games.parquet --season-col year --period-col round \
  --game-col event_id --team-col club_id

Panel base rates

On a complete two-row team-game panel, overall won.mean() is near 0.5 because each game contributes both perspectives. It is not home advantage.

home = panel.loc[panel["is_home"].eq(1)].copy()
print("overall team-row win rate:", panel["won"].mean())
print("home win rate:", home["won"].mean())
print(home.groupby("season")["won"].mean())

Ties, neutral sites, forfeits, incomplete games, and duplicate perspectives can alter this identity. Explain them rather than forcing the expected value.

Missingness

Overall rates can hide an era or provider boundary:

critical = [c for c in ["event_time", "points_for", "points_against", "won"] if c in panel]
print(panel[critical].isna().mean().sort_values(ascending=False))
print(panel.groupby("season")[critical].agg(lambda s: s.isna().mean()))

Classify each important pattern as structurally expected, temporarily unavailable, failed collection/join, not applicable, or unknown. Never silently fill early-history rolling nulls with zero; feature rules must define a prior, missing indicator, or minimum-history threshold.

Targets, distributions, and outliers

print(home["point_diff"].describe(percentiles=[.01, .05, .5, .95, .99]))
print(panel["won"].value_counts(dropna=False, normalize=True))

import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(8, 4.5))
ax.hist(home["point_diff"].dropna(), bins=40, edgecolor="black", alpha=.85)
ax.axvline(0, color="red", linestyle="--")
ax.set(title=f"Home point differential (n={len(home):,})", xlabel="point differential")
ax.grid(alpha=.25, axis="y")
plt.show()

Check scores, margins, counts, and rates against sport semantics. Determine whether extremes are authentic, unit errors, or duplicates. For count targets, report zeros and overdispersion; for players, slice by role and playing-time eligibility before comparing distributions.

Structural breaks

Before pooling seasons, check rule and overtime changes, schedule length, shortened seasons, labor disruptions, expansion/relocation, tracking start dates, provider/schema migrations, definition revisions, and playoff mixing. A real break may require a known-at-T era flag, separate analyses, a sliding window, or exclusion. Do not normalize it away without explanation.

Leakage scouts

For a pre-event task, current-event outcomes and summaries are legal labels but illegal predictors: current score, margin, result, yards, EPA, win probability, box-score totals, final-season aggregates, and post-event participation.

suspects = {"points_for", "points_against", "won", "point_diff", "final_score"}
print("present suspects:", sorted(suspects.intersection(panel.columns)))

Presence is not failure; use as a pre-event feature is. Keep candidate features separate and send them through feature-rules and leakage-audit.

Automated panel report

panel_report.py reads user-owned CSV, Parquet, JSON, JSONL, or NDJSON and expects season, game_id, team, is_home, and binary won, with mapping flags. Install pandas; Parquet also needs pyarrow or fastparquet.

python /path/to/eda-sports/scripts/panel_report.py \
  --input games.parquet --out data/eda.json
python /path/to/eda-sports/scripts/panel_report.py \
  --input games.csv --out data/eda.json --season-col year \
  --game-col event_id --team-col club --home-col home_flag --outcome-col win

Do not guess missing columns. Stop or create an explicit reviewed mapping. The report uses only the documented EDA decisions: GO (status 0), REPAIR (status 1), or STOP (status 2). A game without exactly two distinct team rows and exactly one home row, a null natural key, or a duplicate (game, team) key is structurally incompatible with this helper and returns STOP. Missing outcomes return REPAIR; they may be legitimate future events, but must be separated from completed-event analysis. GO still requires the separate time and leakage reviews described above.

Red flags

Symptom	Interpretation	Action
duplicate natural keys	mixed grain or repeated records	inspect groups; repair cause
wrong rows per contest	incomplete/mislabeled panel	rebuild or restate grain
entity count jump	real expansion or ID churn	verify and document crosswalk
empty periods	incomplete pull or active season	quantify and reload if unexpected
constant scores/target	corrupt source or filter	stop and verify source
overall win rate near .5 on team-game	expected pairing	use one perspective for game claims
early form null	no history	specify prior/minimum history
near-perfect first model	leakage until disproved	stop; run leakage-audit

EDA note template

Question / target / decision time T:
Dataset, source, retrieval/version:
Grain and natural key:
Population and period:
Rows / games / entities:
Key and rows-per-game findings:
Coverage gaps:
Missingness findings:
Target, distribution, and slice findings:
Structural breaks:
Leakage suspects:
Repairs performed / remaining limitations:
Artifacts and plots:
Decision: GO | REPAIR | STOP
Reasons and next action:

Worked example

For a two-season pre-game team win project: declare (game_id, team); run both helpers; confirm two rows per completed game and expected period coverage; measure home win rate on home rows; separate future scheduled games from failed result joins; mark current score/result fields target-only; record era/provider breaks; and issue GO only for the supported population.

Integrity and resource routing

1. State grain before metrics; never model through unexplained duplicate keys.
2. Never hide incomplete periods, missing entities, or repairs.
3. Compare base rates on the correct independent unit.
4. Preserve raw data and separate outcomes from feature candidates.
5. Use references/eda_checklist.md for completeness, references/grain_guide.md for grain/key questions, and references/red_flags.md for diagnosis.
6. Run scripts/coverage_table.py for coverage and scripts/panel_report.py for a machine-readable team-game summary.