Walk forward guard

Skill toilact/godchi/skills/walk-forward-guard

Use when splitting data for training/validation, writing a backtest, evaluating a time-series or trading model, or computing CV scores. Catches random shuffles, k-fold on time-ordered data, and overlapping train/test windows that inflate metrics.From its SKILL.md

Install

npx -y skills add toilact/godchi --skill walk-forward-guard

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.6 KB, as published. Nobody here has run it

Walk-Forward Guard

Overview

Time-series and trading models must be validated walk-forward: train on the past, test on the strictly-later future, never shuffle. Random splits and plain k-fold leak future information into training and produce backtests that don't survive live trading.

When to Use

Trigger on: train_test_split, KFold, cross_val_score, shuffle=True, any backtest loop, choosing a validation scheme, reporting accuracy/Sharpe/PnL on historical data.

Core rule (before → after)

# ❌ WRONG: random split / shuffle on time-ordered data
X_train, X_test = train_test_split(X, test_size=0.2, shuffle=True)   # future in train
scores = cross_val_score(model, X, y, cv=KFold(5, shuffle=True))     # leaks both ways

# ✅ RIGHT: expanding (or rolling) walk-forward, with an embargo gap
from sklearn.model_selection import TimeSeriesSplit
tscv = TimeSeriesSplit(n_splits=5, gap=EMBARGO)   # gap prevents label-window overlap
for tr, te in tscv.split(X):                      # tr always strictly before te
    model.fit(X.iloc[tr], y.iloc[tr])
    preds = model.predict(X.iloc[te])

Checklist

No shuffle. shuffle=False (or omit) on any split of ordered data. No KFold/StratifiedKFold on time series.
Train precedes test in time for every fold — verify max(train_time) < min(test_time).
Embargo / gap between train and test ≥ the label horizon (e.g. a next-day-return label → gap ≥ 1 day) so the train label window doesn't overlap the test window.
No tuning on the test window. Hyperparameter search uses an inner walk-forward split, not the final out-of-sample period.
Report out-of-sample only. Headline metrics come from the held-out future folds, not in-sample.

Common mistakes

Mistake	Why wrong	Fix
`train_test_split(..., shuffle=True)`	future rows in train	`TimeSeriesSplit`
`KFold` on prices	each fold tests on its own past	walk-forward only
no gap between folds	label window overlaps test	`gap=horizon`
tune on the OOS period	overfits the report	inner CV for tuning

Red flags — STOP

"k-fold gives a smoother score" (it's leaking)
80/20 random split on dated rows
Backtest Sharpe looks too good and uses shuffled data
Selecting the model by its test-set score

Pairs with ml-data-leakage-guard. Reply in Vietnamese.