Threading changes the runtime context
Record thread and scheduling choices before comparing inference timing.
Research note · November 14, 2026Research question
How can identical model bytes produce different timing observations on the same host?
Method
- Freeze the model, input and runtime version. Record intra-operation and inter-operation threading settings, execution mode and host core availability for each run.
- Compare a small declared set of thread configurations under a fixed workload. Include warm-up and steady-state observations, and keep concurrent background activity in the record.
- Separate session-construction time from inference time. Report distributions over repeated runs rather than selecting the fastest run as representative.
Records to retain
- Thread settings and runtime options
- Host CPU topology and limits
- Warm-up procedure
- Per-run timing observations
Source basis
ONNX Runtime documents thread-pool controls, affinity and spinning behavior. These are performance-relevant settings even when model and configuration files appear unchanged to a narrow artifact manifest.
Interpretation & limits
A timing difference does not establish an integrity violation. Results would apply to the measured workload and hardware, and no performance result is supplied here.
Next experiments
- Repeat a thread-setting sweep on one documented host.
- Decide which execution options belong in the release policy.
References
Technical reference
Implementation details supporting this note, drawn directly from the maintained documentation. These are technical references, not additional papers or new experimental results. Expand a topic to inspect its method, fields and limits.