Evaluate forward in time
Preserve temporal order when future behavior is the intended target.
Research note · November 14, 2026Research question
Does a random split reveal performance on future data or merely on mixed historical conditions?
Method
- Choose a chronological cutoff and define what information was available at that time. Fit transformations and select thresholds using only the permitted earlier data.
- Evaluate on later intervals and report results by period. Retain changes in prevalence, environment and collection procedure instead of averaging them away.
- Add an explicit gap where neighboring observations would otherwise overlap through a sliding window or delayed label. Record the gap’s rationale and the data it excludes.
Records to retain
- Chronological cutoffs
- Feature-availability times
- Window overlap and gap rules
- Period-specific results
Source basis
Scikit-learn’s time-series cross-validation documentation distinguishes temporal evaluation from ordinary shuffled splits. The direction of information flow matters when a future observation is the target.
Interpretation & limits
A forward split tests the chosen historical periods, not every possible future shift. It can still leak through features or labels created after the prediction time.
Next experiments
- Audit feature timestamps against the decision timestamp.
- Evaluate at more than one historical cutoff before generalizing.
References
Technical reference
Implementation details supporting this note, drawn directly from the maintained documentation. These are technical references, not additional papers or new experimental results. Expand a topic to inspect its method, fields and limits.