What time does criterion measure

110 Views Asked by At

I would like to benchmark my rustcode with criterion, but the information it presents to me do not match up. Here is some of the output I get

Benchmarking big/big decoding: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 306.4s.
big/big decoding        time:   [632.40 ns 724.40 ns 899.80 ns]
                        change: [+0.7649% +16.645% +46.889%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) high mild
  1 (10.00%) high severe

If it took a criterion only 724.40 ns to run the code, why couldn’t it run the code 10 times in 5 seconds. I looked at the outliers in the browser and the one outlier took 1.5 microseconds to run so this could not be the problem.
Here is the benchmark code:

fn big_benchmark(c: &mut Criterion) {
    let file = std::fs::File::open("../../test_big_file").expect("could not open file : ( ");
    let mut output_file =
        std::fs::File::create("../../test_big_file_normal.txt").expect("Could not create file");
    let mut group = c.benchmark_group("big");

    group
        .sample_size(10)
        .warm_up_time(std::time::Duration::from_secs(1))
        .measurement_time(std::time::Duration::from_secs(5));

    let mut base = Decoder::new(file, output_file, EncoderType::Lossy);
    group.bench_function("big decoding", |b| b.iter(|| base.decode().unwrap()));
    group.finish();
}
0

There are 0 best solutions below