Skip to research
Vraelis Research
All researchResearch methods / Reference note and proposed protocol

Evaluate forward in time

Preserve temporal order when future behavior is the intended target.

Research note · November 14, 2026

Research question

Does a random split reveal performance on future data or merely on mixed historical conditions?

Method

  1. Choose a chronological cutoff and define what information was available at that time. Fit transformations and select thresholds using only the permitted earlier data.
  2. Evaluate on later intervals and report results by period. Retain changes in prevalence, environment and collection procedure instead of averaging them away.
  3. Add an explicit gap where neighboring observations would otherwise overlap through a sliding window or delayed label. Record the gap’s rationale and the data it excludes.

Records to retain

  • Chronological cutoffs
  • Feature-availability times
  • Window overlap and gap rules
  • Period-specific results

Source basis

Scikit-learn’s time-series cross-validation documentation distinguishes temporal evaluation from ordinary shuffled splits. The direction of information flow matters when a future observation is the target.

Interpretation & limits

A forward split tests the chosen historical periods, not every possible future shift. It can still leak through features or labels created after the prediction time.

Next experiments

  • Audit feature timestamps against the decision timestamp.
  • Evaluate at more than one historical cutoff before generalizing.

References

  1. Scikit-learn cross-validation
  2. Scikit-learn common pitfalls

Technical reference

Implementation details supporting this note, drawn directly from the maintained documentation. These are technical references, not additional papers or new experimental results. Expand a topic to inspect its method, fields and limits.

Related notes