Skip to main content

DEFAULT_SAMPLE_ORDER

Constant DEFAULT_SAMPLE_ORDER 

Source
pub(crate) const DEFAULT_SAMPLE_ORDER: SampleOrder;
Expand description

The order a run uses when --sample-order is not given.

Round-robin, on this evidence: five suites an arm, interleaved on one machine, one unmodified binary against itself. Over the eighteen rows the order actually governs, the median disagreement between two suites fell from 0.61% to 0.45% and its 90th percentile from 1.97% to 1.67%, thirteen of eighteen rows improved, and the suite took the same time. That is a quarter of the noise and not a fix; it is the default because it costs nothing, not because it settles anything.

A run at --iterations 1 – which is what CI does – takes exactly the same samples in exactly the same sequence under either order, because one pass over the rows is one sample of each.