Skip to research
Vraelis Research
All researchVraelis Imprint / Reference note and proposed protocol

Validation is tied to a question

Define the intended use before calling a physical model validated.

Research note · November 14, 2026

Research question

What evidence would make a simulation useful for one decision but insufficient for another?

Method

  1. State the decision and observable quantities before choosing a model. A model suitable for relative motion may be unsuitable for peak stress, heat transfer or impact loads.
  2. Identify an independent reference and its uncertainty. Compare model outputs under matched boundary conditions, retaining discrepancies and the range of conditions actually examined.
  3. Separate calibration cases from validation cases. Record any parameter changes made after observing a mismatch so the final evaluation does not quietly reuse the data that tuned the model.

Records to retain

  • Intended use and quantities
  • Reference measurements and uncertainty
  • Calibration and validation split
  • Discrepancies by condition

Source basis

NIST uncertainty guidance emphasizes how measurement results and their uncertainty are expressed. Evaluation guidance on leakage also applies to the logic of keeping model tuning separate from held-out assessment.

Interpretation & limits

This note does not validate Imprint or any included creative. A visually convincing animation is not an independent reference, and agreement on one quantity does not validate all uses.

Next experiments

  • Choose one measurable quantity for a small physical system.
  • Define an acceptance rationale before comparing outputs.

References

  1. NIST uncertainty of measurement results
  2. Scikit-learn common pitfalls
  3. NIST AI Risk Management Framework

Technical reference

Implementation details supporting this note, drawn directly from the maintained documentation. These are technical references, not additional papers or new experimental results. Expand a topic to inspect its method, fields and limits.

Related notes