clockmetric.reportMeasurements - returns an empty report

Question

final class SutTest: XCTestCase {
    var sut: Sut!
    
    override func setUp() {
        super.setUp()
        sut = Sut()      
    }
    
    override func tearDown() {
        sut = nil
        super.tearDown()
    }
    
    func testExample() {
        let clockMetric = XCTClockMetric()
        let start = XCTPerformanceMeasurementTimestamp()
        measure(metrics: [clockMetric]) {
            sut.testing()
        }
        let end = XCTPerformanceMeasurementTimestamp()

        let array = try! clockMetric.reportMeasurements(from: start, to: end)
        array.count // array is empty
    }
}

I want to pull out metrics information, found this method reportMeasurements. But when I try to get it, nothing happens.

I want to get the report that shows in xcode but i want to access it in code. I want to get the average value of the code passage time. How can I get information about metrics in the code?

Answer 1

The reportMeasurements method returns an array of XCTClockMeasurement objects. Each XCTClockMeasurement object represents a single measurement of time. The XCTClockMeasurement object has a duration property that represents the amount of time that was measured.

let clockMetric = XCTClockMetric()
let start = XCTPerformanceMeasurementTimestamp()
measure(metrics: [clockMetric]) {
    sut.testing()
}
let end = XCTPerformanceMeasurementTimestamp()

let measurements = try! clockMetric.reportMeasurements(from: start, to: end)
let averageDuration = measurements.reduce(0) { $0 + $1.duration } / measurements.count

averageDuration variable will contain the average amount of time that was spent executing the sut.testing() method.

import XCTest

class SutTest: XCTestCase {
    var sut: Sut!

    override func setUp() {
        super.setUp()
        sut = Sut()
    }

    override func tearDown() {
        sut = nil
        super.tearDown()
    }

    func testExample() {
        let clockMetric = XCTClockMetric()
        let start = XCTPerformanceMeasurementTimestamp()
        measure(metrics: [clockMetric]) {
            sut.testing()
        }
        let end = XCTPerformanceMeasurementTimestamp()

        let measurements = try! clockMetric.reportMeasurements(from: start, to: end)
        let averageDuration = measurements.reduce(0) { $0 + $1.duration } / measurements.count

        XCTAssertEqual(averageDuration, 1.2345, accuracy: 0.0001)
    }
}

class Sut {
    func testing() {
        // Do some work
    }
}

Answer 2

You are not supposed to capture the start and end times yourself and compare manually. Use the metric like this:

func testExample() {
    measure(metrics: [XCTClockMetric()]) {
        sut.testing()
    }
}

Open the Performance Result panel after running the test a few times. There you can edit the acceptable average duration (Baseline) and the allowable maximum deviation (STDDEV). The test will pass only if the measured duration is within the specified range.

You don't want to hard-code the time values anyway. Performance depends very much on the execution environment. If you could hard-code it and set the expected average time for your own test machine, with very small deviation, then the test will likely constantly fail on another test machine.

reportMeasurements(from:to:) is meant to be accessed by development tools. You would have to implement it when writing your own custom XCTMetric, for example.