Closed Bug 1893863 Opened 1 year ago Closed 11 months ago

Intermittent Error while validating PERFHERDER_DATA | single tracking bug

Categories

(Testing :: Talos, defect, P5)

Product:
Testing
Component:
Talos
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: intermittent-bug-filer, Unassigned)

Details

(Keywords: intermittent-failure, intermittent-testcase)

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=456165443&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BqEVyZGbQOesyV663-5ezA/runs/0/artifacts/public/logs/live_backing.log

SUITE-END | took 564s
[task 2024-04-28T10:32:29.252Z] 10:32:29     INFO -  Completed test suite (00:09:24)
[task 2024-04-28T10:32:29.258Z] 10:32:29     INFO -  PERFHERDER_DATA: {"application": {"name": "firefox", "version": "127.0a1"}, "framework": {"name": "talos"}, "suites": [{"name": "basic_compositor_video", "extraOptions": ["e10s", "stylo", "fission", "webrender-sw"], "subtests": [{"name": "240p.120fps.mp4_scale_1_startup", "value": 1.716632420091324, "replicates": [1.6903657850309508, 1.6765848214285712, 1.7046223736513346, 1.7096068376068376, 1.70169779286927, 1.6714699331848553, 1.716632420091324, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "240p.120fps.mp4_scale_1_inclip", "value": 1.7397391304347825, "replicates": [1.6674, 1.7144075021312877, 1.7361764705882352, 1.7333968804159443, 1.7397391304347825, 1.6950338983050846, 1.7273402417962005, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "240p.120fps.mp4_scale_1.1_startup", "value": 1.7538120256859309, "replicates": [1.878747651847214, 1.7373364215402434, 1.7146636259977197, 1.7538120256859309, 1.7175157412707498, 1.748211997670355, 1.7521982507288631, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "240p.120fps.mp4_scale_1.1_inclip", "value": 2.8736411332633787, "replicates": [1.7250946643717724, 1.6908706677937448, 1.695861486486487, 1.6752093802345065, 1.7377469670710577, 1.7160548885077187, 2.8736411332633787, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "240p.120fps.mp4_scale_2_startup", "value": null, "replicates": [1.7353090699017908, 1.7635017626321972, 1.7379676674364894, 1.734968336211859, 1.7404640371229696, 1.800071985602879, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "240p.120fps.mp4_scale_2_inclip", "value": null, "replicates": [1.7084380305602724, 1.6965733672603904, 1.696898305084746, 1.6984889643463494, 1.690152027027028, 1.6908600337268134, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "480p.60fps.webm_scale_1_startup", "value": null, "replicates": [3.3346947835738043, 3.3369966629588443, 3.345144766146992, 3.3398665183537255, 3.345841694537346, 3.340935412026728, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "480p.60fps.webm_scale_1_inclip", "value": null, "replicates": [3.3336106489184694, 3.328438538205976, 3.3335000000000035, 3.329668874172184, 3.348106312292361, 3.3493645484949806, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "480p.60fps.webm_scale_1.1_startup", "value": null, "replicates": [3.3460869565217375, 3.3337624861265276, 3.335377777777778, 3.34, 3.369418344519014, 3.339088888888889, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "480p.60fps.webm_scale_1.1_inclip", "value": null, "replicates": [3.333079470198676, 3.3400998336106453, 3.332292358803988, 3.3333001658374783, 3.335673876871883, 3.3314143094841935, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "480p.60fps.webm_scale_2_startup", "value": null, "replicates": [3.340935412026726, 3.3558212290502825, 3.3442491657397104, 3.3444816053511706, 3.3487402452619883, 3.352790178571427, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "480p.60fps.webm_scale_2_inclip", "value": null, "replicates": [3.3356333333333352, 3.332947019867547, 3.3341763727121467, 3.3335333333333317, 3.335433333333331, 3.333533333333338, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "1080p.60fps.mp4_scale_1_startup", "value": null, "replicates": [3.3464882943143803, 3.36223964165733, 3.3797525309336356, 3.3450278086763072, 3.3310543840177527, 3.388036117381489, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "1080p.60fps.mp4_scale_1_inclip", "value": null, "replicates": [3.330881863560733, 3.3286189683860234, 3.331343283582082, 3.335614617940201, 3.330199335548183, 3.3335215946843832, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "1080p.60fps.mp4_scale_1.1_startup", "value": null, "replicates": [3.389988700564971, 3.348415178571429, 3.3421380846325226, 3.347424749163879, 3.3575391498881424, 3.355061452513965, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "1080p.60fps.mp4_scale_1.1_inclip", "value": null, "replicates": [3.335514950166106, 3.3353333333333284, 3.3350166112956816, 3.328252911813642, 3.329550748752076, 3.333820598006643, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "1080p.60fps.mp4_scale_2_startup", "value": null, "replicates": [3.34236080178174, 3.351517857142856, 3.3413808463251646, 3.3686226203807395, 3.3727415730337054, 3.3644147157190605, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}, {"name": "1080p.60fps.mp4_scale_2_inclip", "value": null, "replicates": [3.335149501661121, 3.3341999999999947, 3.331603305785116, 3.33373333333333, 3.330848585690522, 3.3304318936877046, null, null, null, null, null, null], "lowerIsBetter": true, "alertThreshold": 2.0, "shouldAlert": false, "unit": "ms/frame"}], "shouldAlert": true, "value": null, "lowerIsBetter": true, "alertThreshold": 2.0}]}
[task 2024-04-28T10:32:29.293Z] 10:32:29     INFO - Return code: 0
[task 2024-04-28T10:32:29.345Z] 10:32:29     INFO - Validating PERFHERDER_DATA against /opt/worker/tasks/task_171429889302925/mozharness/external_tools/performance-artifact-schema.json
[task 2024-04-28T10:32:29.348Z] 10:32:29    ERROR - Error while validating PERFHERDER_DATA
[task 2024-04-28T10:32:29.348Z] 10:32:29    ERROR - Traceback (most recent call last):
[task 2024-04-28T10:32:29.348Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.348Z] 10:32:29    ERROR -   File "/opt/worker/tasks/task_171429889302925/mozharness/mozharness/mozilla/testing/talos.py", line 838, in _validate_treeherder_data
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -     jsonschema.validate(data, schema)
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -   File "/opt/worker/tasks/task_171429889302925/build/venv/lib/python3.11/site-packages/jsonschema/validators.py", line 478, in validate
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -     cls(schema, *args, **kwargs).validate(instance)
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -   File "/opt/worker/tasks/task_171429889302925/build/venv/lib/python3.11/site-packages/jsonschema/validators.py", line 123, in validate
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -     raise error
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - jsonschema.exceptions.ValidationError: None is not of type 'number'
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - Failed validating 'type' in schema['properties']['suites']['items']['properties']['subtests']['items']['properties']['value']:
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -     {'description': 'Summary value for subtest',
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -      'maximum': 1000000000000.0,
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -      'minimum': -1000000000000.0,
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -      'title': 'Subtest value',
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -      'type': 'number'}
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - On instance['suites'][0]['subtests'][4]['value']:
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR -     None
[task 2024-04-28T10:32:29.349Z] 10:32:29    ERROR - 
[task 2024-04-28T10:32:29.349Z] 10:32:29  WARNING - setting return code to 1
[task 2024-04-28T10:32:29.349Z] 10:32:29     INFO - Running post-action listener: _package_coverage_data
[task 2024-04-28T10:32:29.349Z] 10:32:29     INFO - Running post-action listener: _resource_record_post_action
[task 2024-04-28T10:32:29.350Z] 10:32:29     INFO - Running post-action listener: process_java_coverage_data
[task 2024-04-28T10:32:29.350Z] 10:32:29     INFO - [mozharness: 2024-04-28 10:32:29.350024Z] Finished run-tests step (success)
[task 2024-04-28T10:32:29.350Z] 10:32:29     INFO - Running post-run listener: _resource_record_post_run
[task 2024-04-28T10:32:29.545Z] 10:32:29     INFO - Total resource usage - Wall time: 602s; CPU: 21%; Read bytes: 94617600; Write bytes: 714104832; Read time: 1262; Write time: 1651
[task 2024-04-28T10:32:29.545Z] 10:32:29     INFO - TinderboxPrint: CPU usage<br/>20.5%
[task 2024-04-28T10:32:29.545Z] 10:32:29     INFO - TinderboxPrint: I/O read bytes / time<br/>94,617,600 / 1,262
[task 2024-04-28T10:32:29.545Z] 10:32:29     INFO - TinderboxPrint: I/O write bytes / time<br/>714,104,832 / 1,651
[task 2024-04-28T10:32:29.545Z] 10:32:29     INFO - TinderboxPrint: CPU idle<br/>5,586.2 (77.4%)

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.