Open
Bug 1962881
Opened 17 days ago
Updated 1 day ago
Intermittent raptor-main Critical: TEST-UNEXPECTED-FAIL: no raptor test results were found for youtube-playback-h264-720p60
Categories
(Testing :: mozperftest, defect, P5)
Testing
mozperftest
Tracking
(Not tracked)
NEW
People
(Reporter: intermittent-bug-filer, Unassigned)
Details
(Keywords: intermittent-failure)
Filed by: nfay [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=505622627&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/fK1_93x-QjW3PLfG7Caowg/runs/0/artifacts/public/logs/live_backing.log
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - raptor-browsertime Info: Wrote data to blobber_upload_dir/browsertime-results/youtube-playback-h264-720p60
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - raptor-browsertime-android Info: removing reverse socket connections
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - raptor-browsertime Info: Test support class is cleaning up...
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: retrieving browsertime test results
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: found browsertime results at /builds/task_174571384937507/workspace/build/blobber_upload_dir/browsertime-results/youtube-playback-h264-720p60/browsertime.json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: {'here': '/builds/task_174571384937507/workspace/build/tests/raptor/raptor/tests/benchmarks', 'alert_threshold': '2.0', 'expose_browser_profiler': True, 'lower_is_better': True, 'owner': 'PerfTest Team', 'page_cycles': '20', 'page_timeout': 3240000.0, 'subtest_lower_is_better': True, 'subtest_unit': 'score', 'type': 'benchmark', 'unit': 'score', 'use_live_sites': 'true', 'preferences': {'media.autoplay.default': 0, 'media.autoplay.ask-permission': False, 'media.autoplay.blocking_policy': 0, 'media.allowed-to-play.enabled': True, 'media.block-autoplay-until-in-foreground': False}, 'apps': 'firefox, chrome, geckoview, fenix, refbrow, chrome-m', 'test_url': 'https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&tests=38&raptor=true&exclude=1,2&muted=true&command=run', 'support_class': <browsertime_benchmark.py.BenchmarkSupport object at 0x7fda1dd54370>, 'browsertime_args': '--pageCompleteWaitTime=10 --pageCompleteCheckPollTimeout=10 --browsertime.benchmark_wait_time=500', 'name': 'youtube-playback-h264-720p60', 'manifest': '/builds/task_174571384937507/workspace/build/tests/raptor/raptor/tests/benchmarks/youtube-playback.toml', 'manifest_relpath': 'tests/benchmarks/youtube-playback.toml', 'path': '/builds/task_174571384937507/workspace/build/tests/raptor/raptor/tests/benchmarks/youtube-playback-h264-720p60', 'relpath': 'tests/benchmarks/youtube-playback-h264-720p60', 'ancestor_manifest': 'raptor.toml', 'expected': 'pass', 'browser_cycles': 1, 'cold': False, 'expected_browser_cycles': 1, 'browser_cycle': 1, 'playback': None}
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: parsing results from browsertime json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: {'here': '/builds/task_174571384937507/workspace/build/tests/raptor/raptor/tests/benchmarks', 'alert_threshold': '2.0', 'expose_browser_profiler': True, 'lower_is_better': True, 'owner': 'PerfTest Team', 'page_cycles': '20', 'page_timeout': 3240000.0, 'subtest_lower_is_better': True, 'subtest_unit': 'score', 'type': 'benchmark', 'unit': 'score', 'use_live_sites': 'true', 'preferences': {'media.autoplay.default': 0, 'media.autoplay.ask-permission': False, 'media.autoplay.blocking_policy': 0, 'media.allowed-to-play.enabled': True, 'media.block-autoplay-until-in-foreground': False}, 'apps': 'firefox, chrome, geckoview, fenix, refbrow, chrome-m', 'test_url': 'https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&tests=38&raptor=true&exclude=1,2&muted=true&command=run', 'support_class': <browsertime_benchmark.py.BenchmarkSupport object at 0x7fda1dd54370>, 'browsertime_args': '--pageCompleteWaitTime=10 --pageCompleteCheckPollTimeout=10 --browsertime.benchmark_wait_time=500', 'name': 'youtube-playback-h264-720p60', 'manifest': '/builds/task_174571384937507/workspace/build/tests/raptor/raptor/tests/benchmarks/youtube-playback.toml', 'manifest_relpath': 'tests/benchmarks/youtube-playback.toml', 'path': '/builds/task_174571384937507/workspace/build/tests/raptor/raptor/tests/benchmarks/youtube-playback-h264-720p60', 'relpath': 'tests/benchmarks/youtube-playback-h264-720p60', 'ancestor_manifest': 'raptor.toml', 'expected': 'pass', 'browser_cycles': 1, 'cold': False, 'expected_browser_cycles': 1, 'browser_cycle': 1, 'playback': None}
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: parsed new benchmark result: {'bt_ver': '22.10.1', 'browser': {'cpuBenchmark': 61, 'userAgent': 'Mozilla/5.0 (Android 14; Mobile; rv:139.0) Gecko/139.0 Firefox/139.0', 'windowSize': '980x1651'}, 'url': ('https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&tests=38&raptor=true&exclude=1,2&muted=true&command=run,',), 'name': 'youtube-playback-h264-720p60', 'measurements': {'youtube-playbackperf-hfr-test': [[{'PlaybackPerfH264720p60fps@1X': {'fail': 0, 'decodedFrames': 925, 'droppedFrames': 0}}]]}, 'statistics': {}, 'custom_data': True, 'type': 'benchmark', 'unit': 'score', 'lower_is_better': True, 'alert_threshold': '2.0', 'cold': False, 'min_back_window': None, 'max_back_window': None, 'fore_window': None, 'alert_change_type': None, 'subtest_lower_is_better': True, 'subtest_unit': 'score', 'extra_options': ['fission', 'webrender'], 'tags': [], 'support_class': <browsertime_benchmark.py.BenchmarkSupport object at 0x7fda1dd54370>, 'gather_cpuTime': None}
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-output Info: preparing browsertime results for output
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-output Info: PERFHERDER_DATA: {"framework": {"name": "browsertime"}, "suites": [{"name": "youtube-playback-h264-720p60", "type": "benchmark", "extraOptions": ["fission", "webrender"], "tags": ["fission", "webrender"], "lowerIsBetter": true, "unit": "score", "alertThreshold": 2.0, "subtests": [{"name": "H264720p60fps@1X_%_dropped_frames", "unit": "score", "lowerIsBetter": true, "replicates": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 100.0, 100.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "value": 0.0}, {"name": "H264720p60fps@1X_decoded_frames", "unit": "score", "lowerIsBetter": false, "replicates": [925, 906, 901, 907, 909, 907, 906, 914, 902, 906, 0, 0, 908, 905, 928, 921, 917, 912, 906, 909], "value": 907.0}, {"name": "H264720p60fps@1X_dropped_frames", "unit": "score", "lowerIsBetter": true, "replicates": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "value": 0.0}, {"unit": "ms", "alertThreshold": 2.0, "lowerIsBetter": true, "name": "cpuTime", "replicates": [26000, 22000, 17000, 19000, 19000, 19000, 21000, 20000, 21000, 32000, 13000, 11000, 19000, 19000, 19000, 19000, 20000, 19000, 20000, 19000], "shouldAlert": true, "value": 19700.0}, {"unit": "uWh", "alertThreshold": 2.0, "lowerIsBetter": true, "name": "powerUsage", "replicates": [5870.83, 4934.71, 3915.78, 4625.65, 4346.86, 4766.35, 5359.38, 4405.61, 5524.02, 12250.98, 26851.51, 20700.05, 3941.35, 4177.39, 4042.51, 4306.12, 4325.53, 3947.83, 4524.9, 3954.67], "shouldAlert": true, "value": 6838.601}, {"unit": "ms", "alertThreshold": 2.0, "lowerIsBetter": true, "name": "wallclock-for-tracking-only", "replicates": [14664.96, 12112.09, 11099.86, 11681.43, 11787.64, 11635.02, 12353.51, 11831.24, 12411.26, 35120.31, 95118.41, 95792.26, 12037.87, 11952.29, 12017.5, 12698.79, 12062.02, 11573.81, 12732.42, 11583.55], "shouldAlert": true, "value": 21613.312}], "value": 0.0}], "application": {"name": "fenix"}}
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-output Info: results can also be found locally at: /builds/task_174571384937507/workspace/build/raptor.json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: Validating PERFHERDER_DATA against /builds/task_174571384937507/workspace/mozharness/external_tools/performance-artifact-schema.json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-results-handler Info: Writing video jobs and application data {'jobs': [{'browsertime_json_path': 'browsertime-results/youtube-playback-h264-720p60/browsertime.json', 'test_name': 'youtube-playback-h264-720p60', 'tags': [], 'extra_options': ['fission', 'webrender'], 'accept_zero_vismet': False}], 'application': {'name': 'fenix'}, 'extra_options': ['fission', 'webrender']} into /builds/task_174571384937507/workspace/build/blobber_upload_dir/browsertime-results/jobs.json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-support-class Warning: Some tests failed.
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-support-class Warning: Youtube sub-test FAILED: PlaybackPerfH264720p60fps@1X test Failed. decodedFrames 0 droppedFrames 0.
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-support-class Warning: Youtube sub-test FAILED: PlaybackPerfH264720p60fps@1X test Failed. decodedFrames 0 droppedFrames 0.
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - perftest-support-class Warning: Youtube playback sub-tests failed!!! Not submitting results to perfherder!
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - raptor-perftest Info: Removing temporary directory: /tmp/tmpojvp8d_y
[task 2025-04-27T00:44:33.266Z] 00:44:33 ERROR - raptor-main Critical: TEST-UNEXPECTED-FAIL: no raptor test results were found for youtube-playback-h264-720p60
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Return code: 1
[task 2025-04-27T00:44:33.266Z] 00:44:33 WARNING - setting return code to 1
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Killing logcat pid 832.
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Copying Raptor results to upload dir:
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - /builds/task_174571384937507/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Copying raptor results from /builds/task_174571384937507/workspace/build/raptor.json to /builds/task_174571384937507/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Running post-action listener: _package_coverage_data
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Running post-action listener: _resource_record_post_action
[task 2025-04-27T00:44:33.266Z] 00:44:33 INFO - Running post-action listener: process_java_coverage_data
[task 2025-04-27T00:44:41.918Z] 00:44:33 INFO - Running post-action listener: stop_device
[task 2025-04-27T00:44:41.918Z] 00:44:35 INFO - /data/tombstones/tombstone_03 deleted
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - /data/tombstones/tombstone_03.pb deleted
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - Killing logcat pid 832.
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - [mozharness: 2025-04-27 00:44:36.043774Z] Finished run-tests step (success)
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - Running post-run listener: _resource_record_post_run
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - Total resource usage - Wall time: 764s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - TinderboxPrint: I/O read bytes / time<br/>0 / 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - TinderboxPrint: I/O write bytes / time<br/>0 / 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - TinderboxPrint: Swap in / out<br/>0 / 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - install-chrome-android - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - install-chromium-android - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - install-chromium-distribution - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - install - Wall time: 23s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 INFO - run-tests - Wall time: 739s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2025-04-27T00:44:41.918Z] 00:44:36 WARNING - returning nonzero exit status 1
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
You need to log in
before you can comment on or make changes to this bug.
Description
•