Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Difference of values in duration in HTML report and xctest log #329

Open
jbatra-wondermentapps opened this issue May 26, 2023 · 0 comments

Comments

@jbatra-wondermentapps
Copy link

jbatra-wondermentapps commented May 26, 2023

The HTML report and test-case logs have different values for duration

Is this a bug or some misunderstanding for interpretation?

Please route me to the right approach to read the logs.

Testing started
2023-05-26 12:24:58.207271+0530 PSSwiftExpressUITests-Runner[1191:259871] Running tests...
Test Suite 'Selected tests' started at 2023-05-26 12:24:58.269
Test Suite 'PSSwiftExpressUITests.xctest' started at 2023-05-26 12:24:58.270
Test Suite 'PerformanceTests' started at 2023-05-26 12:24:58.270
Test Case '-[PSSwiftExpressUITests.PerformanceTests testColdAppLaunchTime]' started.
    t =     0.00s Start Test at 2023-05-26 12:24:58.270
    t =     0.00s Set Up
executing: -[PerformanceTests testColdAppLaunchTime]
Battery before start:  0.96
RAM before start:  607993856
Free Disk space before start:  45.87 GB
    t =     0.01s Open com.adobe.PSMobile
    t =     0.01s     Launch com.adobe.PSMobile
    t =     0.01s         Terminate com.adobe.PSMobile:1168
    t =    21.39s         Setting up automation session
    t =    22.25s         Wait for com.adobe.PSMobile to idle
    t =    35.23s Open com.adobe.PSMobile
    t =    35.23s     Launch com.adobe.PSMobile
    t =    35.23s         Terminate com.adobe.PSMobile:1194
    t =    37.59s         Setting up automation session
    t =    37.70s         Wait for com.adobe.PSMobile to idle
    t =    49.34s Open com.adobe.PSMobile
    t =    49.34s     Launch com.adobe.PSMobile
    t =    49.34s         Terminate com.adobe.PSMobile:1198
    t =    51.61s         Setting up automation session
    t =    51.71s         Wait for com.adobe.PSMobile to idle
    t =    63.30s Open com.adobe.PSMobile
    t =    63.30s     Launch com.adobe.PSMobile
    t =    63.30s         Terminate com.adobe.PSMobile:1200
    t =    65.75s         Setting up automation session
    t =    65.85s         Wait for com.adobe.PSMobile to idle
    t =    77.44s Open com.adobe.PSMobile
    t =    77.44s     Launch com.adobe.PSMobile
    t =    77.44s         Terminate com.adobe.PSMobile:1203
    t =    79.95s         Setting up automation session
    t =    80.05s         Wait for com.adobe.PSMobile to idle
    t =    91.65s Open com.adobe.PSMobile
    t =    91.65s     Launch com.adobe.PSMobile
    t =    91.65s         Terminate com.adobe.PSMobile:1205
    t =    94.35s         Setting up automation session
    t =    94.37s         Wait for com.adobe.PSMobile to idle
/Users/usr/Documents/Adobe/Example/example-ios/PSSwiftExpressUITests/tests/PerformanceTests.swift:315: Test Case '-[PSSwiftExpressUITests.PerformanceTests testColdAppLaunchTime]' measured [Duration (AppLaunch), s] average: 0.436, relative standard deviation: 3.916%, values: [0.468968, 0.420271, 0.426677, 0.432267, 0.431298], performanceMetricID:com.apple.dt.XCTMetric_ApplicationLaunch-AppLaunch.duration, baselineName: "Local Baseline", baselineAverage: 2.786, polarity: prefers smaller, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.000, maxStandardDeviation: 0.000
    t =   105.97s Tear Down
    t =   105.97s     -[PerformanceTests testColdAppLaunchTime]
    t =   107.15s     Terminate com.adobe.PSMobile:1206
Battery after end:  0.96
RAM after end:  590168064
Free Disk Space after end:  45.87 GB
Test Case '-[PSSwiftExpressUITests.PerformanceTests testColdAppLaunchTime]' passed (108.186 seconds).
Test Suite 'PerformanceTests' passed at 2023-05-26 12:26:46.458.
	 Executed 1 test, with 0 failures (0 unexpected) in 108.186 (108.188) seconds
Test Suite 'PSSwiftExpressUITests.xctest' passed at 2023-05-26 12:26:46.460.
	 Executed 1 test, with 0 failures (0 unexpected) in 108.186 (108.190) seconds
Test Suite 'Selected tests' passed at 2023-05-26 12:26:46.461.
	 Executed 1 test, with 0 failures (0 unexpected) in 108.186 (108.192) seconds
2023-05-26 12:26:46.379 xcodebuild[95428:2845548] [MT] IDETestOperationsObserverDebug: 113.146 elapsed -- Testing started completed.
2023-05-26 12:26:46.379 xcodebuild[95428:2845548] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2023-05-26 12:26:46.379 xcodebuild[95428:2845548] [MT] IDETestOperationsObserverDebug: 113.146 sec, +113.146 sec -- end

Test session results, code coverage, and logs:
	/Users/usr/Documents/Adobe/Example/Example-ios/TestResults

** TEST SUCCEEDED **

usr@usr-MacBook-Pro Example-ios % xchtmlreport /Users/usr/Documents/Adobe/Example/Example-ios/TestResults.xcresult                                                     

Report successfully created at /Users/usr/Documents/Adobe/Example/Example-ios/index.html
usr@usr-MacBook-Pro Example-ios % xchtmlreport /Users/usr/Documents/Adobe/Example/Example-ios/TestResults.xcresult

HTML report

index 2.html.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant