Closed Bug 1656855 Opened 4 years ago Closed 4 years ago

Perma windows perftest-http3 cloudflare, fb, gsearch jsonschema.exceptions.ValidationError: [']', 's', 'h', 'p', '[', '3', '5', '2', 't', ','] is too long

Categories

(Testing :: mozperftest, defect, P1)

Product:
Testing
Component:
mozperftest
Version:
Default
Type:
defect

Tracking

(firefox-esr68 unaffected, firefox-esr78 unaffected, firefox79 unaffected, firefox80 unaffected, firefox81 fixed)

Status:
RESOLVED FIXED
Milestone:
81 Branch
Tracking Flags:
Tracking Status
firefox-esr68 --- unaffected
firefox-esr78 --- unaffected
firefox79 --- unaffected
firefox80 --- unaffected
firefox81 --- fixed

People

(Reporter: aryx, Assigned: sparky)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

Bug 1656855 - Make perfherder-metrics character escapes OS-specific. r?tarek
47 bytes, text/x-phabricator-request
Details | Review

See https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&group_state=expanded&resultStatus=success%2Csuperseded%2Cusercancel%2Cretry%2Ctestfailed%2Cbusted%2Cexception&tier=1%2C2%2C3&searchStr=perftest&tochange=f6a3b097f8af440a142b02450a2cdaa978fdde9f&selectedTaskRun=VHkMqg2KSVC_E6LROXflDA.0&fromchange=4b6ce69368c336a1d0fba758fb9d6c89650840b4

Log example: https://treeherder.mozilla.org/logviewer.html#?job_id=311829774&repo=mozilla-central

[task 2020-08-03T05:48:21.594Z] [2020-08-03 05:48:21] INFO: Setting up search
[task 2020-08-03T05:48:26.596Z] [2020-08-03 05:48:26] INFO: Start search
[task 2020-08-03T05:48:26.596Z] [2020-08-03 05:48:26] INFO: Start to measure pageload
[task 2020-08-03T05:48:31.772Z] [2020-08-03 05:48:31] INFO: Done
[task 2020-08-03T05:48:31.772Z] [2020-08-03 05:48:31] INFO: Waiting for 25000 milliseconds
[task 2020-08-03T05:48:56.780Z] [2020-08-03 05:48:56] INFO: Going to second page of search results
[task 2020-08-03T05:48:56.780Z] [2020-08-03 05:48:56] INFO: Start to measure pageload
[task 2020-08-03T05:49:06.825Z] [2020-08-03 05:49:06] INFO: https://www.google.com/search?source=hp&ei=36InX6zTH6GpytMP7f-iqAM&q=Python&oq=Python&gs_lcp=CgZwc3ktYWIQAzIFCAAQsQMyAggAMgUIABCxAzIFCC4QsQMyBQgAELEDMgUIABCxAzIFCAAQsQMyBQgAELEDMgUIABCxAzIFCAAQsQNQujhY1ThgvF9oAHAAeACAAYkBiAHkA5IBAzEuM5gBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwjsvvznqP7qAhWhlHIEHe2_CDUQ4dUDCAg&uact=5 has been tested before within the same run, it will get an extra query parameter named browsertime_run. Make sure to use alias to keep track of the URLs
[task 2020-08-03T05:49:07.568Z] [2020-08-03 05:49:07] INFO: https://www.google.com/search?source=hp&ei=36InX6zTH6GpytMP7f-iqAM&q=Python&oq=Python&gs_lcp=CgZwc3ktYWIQAzIFCAAQsQMyAggAMgUIABCxAzIFCC4QsQMyBQgAELEDMgUIABCxAzIFCAAQsQMyBQgAELEDMgUIABCxAzIFCAAQsQNQujhY1ThgvF9oAHAAeACAAYkBiAHkA5IBAzEuM5gBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwjsvvznqP7qAhWhlHIEHe2_CDUQ4dUDCAg&uact=5 TTFB: 119ms DOMContentLoaded: 719ms firstPaint: 160ms Load: 1.25s
[task 2020-08-03T05:49:07.569Z] [2020-08-03 05:49:07] INFO: https://www.google.com/search?source=hp&ei=36InX6zTH6GpytMP7f-iqAM&q=Python&oq=Python&gs_lcp=CgZwc3ktYWIQAzIFCAAQsQMyAggAMgUIABCxAzIFCC4QsQMyBQgAELEDMgUIABCxAzIFCAAQsQMyBQgAELEDMgUIABCxAzIFCAAQsQNQujhY1ThgvF9oAHAAeACAAYkBiAHkA5IBAzEuM5gBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwjsvvznqP7qAhWhlHIEHe2_CDUQ4dUDCAg&uact=5 TTFB: 130ms DOMContentLoaded: 649ms firstPaint: 175ms Load: 1.18s
[task 2020-08-03T05:49:07.573Z] [2020-08-03 05:49:07] INFO: https://www.google.com/search?source=hp&ei=36InX6zTH6GpytMP7f-iqAM&q=Python&oq=Python&gs_lcp=CgZwc3ktYWIQAzIFCAAQsQMyAggAMgUIABCxAzIFCC4QsQMyBQgAELEDMgUIABCxAzIFCAAQsQMyBQgAELEDMgUIABCxAzIFCAAQsQNQujhY1ThgvF9oAHAAeACAAYkBiAHkA5IBAzEuM5gBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwjsvvznqP7qAhWhlHIEHe2_CDUQ4dUDCAg&uact=5 TTFB: 124ms (±0.84ms), firstPaint: 166ms (±1.36ms), DOMContentLoaded: 724ms (±26.91ms), Load: 1.25s (±38.00ms) (20 runs)
[task 2020-08-03T05:49:07.576Z] [2020-08-03 05:49:07] INFO: Wrote data to ..\artifacts\browsertime-results-1
[task 2020-08-03T05:49:07.729Z] Installing browsertime for mach using "C:\Users\task_1596431888\fetches\node\npm.CMD ci --loglevel=error --no-optional --scripts-prepend-node-path"...
[task 2020-08-03T05:49:07.729Z]
[task 2020-08-03T05:49:07.729Z] browsertime installed successfully!
[task 2020-08-03T05:49:07.729Z]
[task 2020-08-03T05:49:07.729Z] NOTE: Your local browsertime binary is at C:\Users\task_1596431888.mozbuild\browsertime\node_modules.bin\browsertime
[task 2020-08-03T05:49:07.729Z]
[task 2020-08-03T05:49:07.729Z] Writing results to C:/Users/task_1596431888/fetches/../artifacts\g-searchstd-output.json
[task 2020-08-03T05:49:07.731Z] Traceback (most recent call last):
[task 2020-08-03T05:49:07.731Z] File "python/mozperftest/mozperftest/runner.py", line 201, in <module>
[task 2020-08-03T05:49:07.731Z] sys.exit(main())
[task 2020-08-03T05:49:07.731Z] File "python/mozperftest/mozperftest/runner.py", line 197, in main
[task 2020-08-03T05:49:07.731Z] run_tests(mach_cmd, **dict(args._get_kwargs()))
[task 2020-08-03T05:49:07.731Z] File "python/mozperftest/mozperftest/runner.py", line 144, in run_tests
[task 2020-08-03T05:49:07.731Z] e.run(metadata)
[task 2020-08-03T05:49:07.731Z] File "C:\Users\task_1596431888\fetches\python\mozperftest\mozperftest\environment.py", line 97, in run
[task 2020-08-03T05:49:07.731Z] metadata = metrics(metadata)
[task 2020-08-03T05:49:07.731Z] File "C:\Users\task_1596431888\fetches\python\mozperftest\mozperftest\layers.py", line 156, in call
[task 2020-08-03T05:49:07.731Z] metadata = layer(metadata)
[task 2020-08-03T05:49:07.731Z] File "C:\Users\task_1596431888\fetches\python\mozperftest\mozperftest\layers.py", line 68, in call
[task 2020-08-03T05:49:07.731Z] metadata = self.run(metadata)
[task 2020-08-03T05:49:07.731Z] File "C:\Users\task_1596431888\fetches\python\mozperftest\mozperftest\metrics\perfherder.py", line 150, in run
[task 2020-08-03T05:49:07.731Z] jsonschema.validate(all_perfherder_data, schema)
[task 2020-08-03T05:49:07.731Z] File "C:\Users\task_1596431888\fetches\third_party\python\jsonschema\jsonschema\validators.py", line 934, in validate
[task 2020-08-03T05:49:07.731Z] raise error
[task 2020-08-03T05:49:07.731Z] jsonschema.exceptions.ValidationError: [']', 's', 'h', 'p', '[', '3', '5', '2', 't', ','] is too long
[task 2020-08-03T05:49:07.731Z]
[task 2020-08-03T05:49:07.731Z] Failed validating 'maxItems' in schema['properties']['suites']['items']['properties']['extraOptions']:
[task 2020-08-03T05:49:07.731Z] {'items': {'maxLength': 100, 'type': 'string'},
[task 2020-08-03T05:49:07.731Z] 'maxItems': 8,
[task 2020-08-03T05:49:07.731Z] 'title': 'Extra options used in running suite',
[task 2020-08-03T05:49:07.731Z] 'type': 'array',
[task 2020-08-03T05:49:07.731Z] 'uniqueItems': True}
[task 2020-08-03T05:49:07.731Z]
[task 2020-08-03T05:49:07.731Z] On instance['suites'][0]['extraOptions']:
[task 2020-08-03T05:49:07.731Z] [']', 's', 'h', 'p', '[', '3', '5', '2', 't', ',']
[fetches 2020-08-03T05:49:07.844Z] removing C:/Users/task_1596431888/fetches

Set release status flags based on info from the regressing bug 1655031

status-firefox79: --- → unaffected
status-firefox80: --- → unaffected
status-firefox81: --- → affected
status-firefox-esr68: --- → unaffected
status-firefox-esr78: --- → unaffected
Assignee: tarek → gmierz2
Severity: -- → S2
Priority: -- → P1
Pushed by gmierz2@outlook.com:
https://hg.mozilla.org/integration/autoland/rev/5b93806618af
Make perfherder-metrics character escapes OS-specific. r=tarek

https://hg.mozilla.org/mozilla-central/rev/5b93806618af

Status: NEW → RESOLVED
Closed: 4 years ago
status-firefox81: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 81 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: