1790755 - Intermitten debug Wdspec 1 task "Log parsing was skipped since the log exceeds the size limit."

Status: RESOLVED FIXED

(Testing :: geckodriver, defect, P5)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Example: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&searchStr=test-windows10-64-2004-qr%2Fdebug-web-platform-tests-wdspec&tochange=a389830fb63fdeb737154791ab1d199cd9082e1d&fromchange=323335d5c537309f05efa246f42867b703a72914&selectedTaskRun=T1zbYF8ISgmvj1oETkkqDQ.0
The test-windows10-64-2004-qr/debug-web-platform-tests-wdspec-1 tasks succeed but the log fails to parse because it's above the 5MB threshold fo the compressed version.

There is a big data base64 blob after these lines:

[task 2022-09-14T05:00:22.352Z] 05:00:22     INFO - PID 1240 | 1663131622351	RemoteAgent	DEBUG	PDF output written to C:\Users\task_166312537297907\AppData\LocalLow\Mozilla\Temp-{f085a535-3206-4d60-9c13-6e7ba75d2b16}\marionette.pdf
[task 2022-09-14T05:00:22.564Z] 05:00:22     INFO - PID 1240 | 1663131622564	Marionette	DEBUG	0 <- [1,79,null,{"value":"JVBERi0xLjUKJbXtrvsKNCAwIG9iago8PCAvTGVuZ3RoIDUgMCBSCiAgIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlCj4+CnN0cmVhbQp4nIVO ... aIVAwFkjL9F0BsxmCwiB1YZCuInHANTL4Fm8DDwAgmmRhlJomCxCfpMDAAAI2vDNQKZW5kc3RyZWFtCmVuZG9iagpzdGFydHhyZWYKMTg3MjQyOAolJUVPRgo="}]

Setting as regression from the pdf.js update in bug 1790394. The fix might be to change what gets logged by geckodriver.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

I'll have a look at bug 1611004 which I just added as blocker for this bug. Goal is to also truncate the payload similar to Marionette by default.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

:calixte, since you are the author of the regressor, bug 1790394, could you take a look? Also, could you set the severity field?

For more information, please visit auto_nag documentation.

Comment 3

[Calixte Denizet (:calixte)]

I don't see how it could be related to this pdf.js update.
:whimboo, do you think there is a correlation here ?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

Not sure at this point. Logging full PDF details is still not great and I'm working to stop this in bug 1611004.

Nevertheless I wonder if the size of the generated PDFs has been increased with this update and causes us to now exceed the error parsing limit.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:whimboo, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

Note that this doesn't happen permanently anymore. I can see it roughly only in 1 of 6 cases.

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This got much more frequent again. Examples:

test-windows10-32-2004-qr/debug-web-platform-tests-wdspec-headless-1 https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=7e6ddb806ce0cdaea5d6cbb3cc21581f8d09c541&selectedJob=395255035&searchStr=test-windows10-32-2004-qr/debug-web-platform-tests-wdspec-headless-1
test-windows10-32-2004-qr/debug-web-platform-tests-wdspec-1 https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=7e6ddb806ce0cdaea5d6cbb3cc21581f8d09c541&selectedJob=395255005&searchStr=test-windows10-32-2004-qr/debug-web-platform-tests-wdspec-1
test-linux1804-64-qr/debug-web-platform-tests-wdspec-headless-1 https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&group_state=expanded&revision=728d2e95159c89f3ddacd40d3e9fecc0c0177474&selectedJob=395249121&searchStr=test-linux1804-64-qr/debug-web-platform-tests-wdspec-headless-1
test-macosx1015-64-qr/debug-web-platform-tests-wdspec-1 https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=51bf1455bb63aa6bd69f39d2a1cbf27f631d0fa3&selectedJob=395218009&searchStr=test-macosx1015-64-qr/debug-web-platform-tests-wdspec-1
test-macosx1015-64-qr/debug-web-platform-tests-wdspec-headless-1 https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=51bf1455bb63aa6bd69f39d2a1cbf27f631d0fa3&selectedJob=395217973&searchStr=test-macosx1015-64-qr/debug-web-platform-tests-wdspec-headless-1

Please have a look what pushed it over the 5MB gzipped parsing limit.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

I landed a huge chunk of new tests via bug bug 1774182 yesterday:

https://treeherder.mozilla.org/jobs?repo=autoland&tochange=77a9f0ad9a10e60fb72724c2a304119589f13d8e&fromchange=7f503b2c8be5ba596187a9e82e39c4e3d0cedc45

Because of that some tests might have shifted between the chunks and most likely the PDF generation tests that are running in Wd1 now are causing more log output for Wd1. I'll focus on bug 1611004 tomorrow to have that issue fixed soon.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

Aryx this should be fixed and I cannot find a log where the failure lines haven't been parsed. Can you please check? Thanks.

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Latest wdspec headless observed Friday early evening European time.