1899140 - source-test-puppeteer-puppeteer-with-bidi logs too large to be parse by Treeherder

Status: RESOLVED FIXED

(Remote Protocol :: WebDriver BiDi, defect, P2)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

The source-test-puppeteer-puppeteer-with-bidi task logs are too large to be parsed by Treeherder (threshold: 5 MB gzipped). Failure lines are not shown, hence the log size should be reduced. Task example

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is a regression from the last Puppeteer sync as done on bug 1897183.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1897183

:Sasha, since you are the author of the regressor, bug 1897183, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This happens most likely because of the test AriaQueryHandler queryAllArray $$eval should handle many elements (ariaqueryhandler.spec.js).

Here 10.000 elements of type button are getting created within a single evaluate call, then retrieved via browsingContext.locateNodes as sharedId, evaluated individually to get a handle, evaluated once to call reduce, and finally the handles are disowned individually.

 0:05.23 pid:22179 1716817055602	RemoteAgent	DEBUG	WebDriverBiDiConnection 7afd76bf-0fcd-4335-a43e-55b729ebe1de -> {"id":156,"method":"script.callFunction","params":{"functionDeclaration":"node => {\n            return node;\n        }\n//# sourceURL=pptr:internal\n","awaitPromise":true,"target":{"context":"0a3ff222-04e6-43df-aec1-8476cd80f724"},"arguments":[{"type":"node","sharedId":"ea2b5108-5e80-4095-a950-bdf65e7def14","value":{"nodeType":1,"localName":"button","namespaceURI":"http://www.w3.org/1999/xhtml","childNodeCount":1,"attributes":{},"shadowRoot":null}}],"resultOwnership":"root","userActivation":true,"serializationOptions":{"maxObjectDepth":0,"maxDomDepth":0}}}
 0:06.41 pid:22179 1716817056164	RemoteAgent	DEBUG	WebDriverBiDiConnection 7afd76bf-0fcd-4335-a43e-55b729ebe1de <- {"type":"success","id":156,"result":{"realm":"c1657cae-c1a6-4ad8-a140-bb45bace99c3","type":"success","result":{"type":"node","handle":"ced784db-2d5f-429c-a087-7e048ce332f5","sharedId":"ea2b5108-5e80-4095-a950-bdf65e7def14","value":{"nodeType":1,"localName":"button","namespaceURI":"http://www.w3.org/1999/xhtml","childNodeCount":1,"attributes":{},"shadowRoot":null}}}}
[..]
 0:06.79 pid:22179 1716817056972	RemoteAgent	DEBUG	WebDriverBiDiConnection 7afd76bf-0fcd-4335-a43e-55b729ebe1de -> {"id":10158,"method":"script.disown","params":{"target":{"context":"0a3ff222-04e6-43df-aec1-8476cd80f724"},"handles":["ced784db-2d5f-429c-a087-7e048ce332f5"]}}

Given that we have trace logs enabled this is causing quite a lot of output. So I've two questions:

1. Are that many elements really needed?
2. Evaluating nodes in Puppeteer always requires a handle?
3. Why cannot all buttons dismissed at once by calling script.disown a single time?

If we cannot reduce the output we will have to most likely reduce the log output from trace to debug or even info, or starting to chunk the tests.

Alex, mind to give some feedback? Thanks.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As discussed with Alex on Matrix yesterday the test will stay as is. That means we have to find another solution for us, which probably brings us down to these two:

1. Add chunking to the pup (wd) jobs . It may work as long as this test isn't part of the job that has some other test with quite a lot of logging.
2. Don't run with trace logs by default - even debug might be too much (we should check). Here we might not have the chance to easily investigate failures

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As discussed lets reduce the number of elements that test is using so that we don't miss coverage. I'll file a follow-up bug so that we can work on a better fix for logging and at the same time get the log summaries for failure classification working again.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1899140 - [puppeteer] Skip test "$$eval should handle many elements" which is causing the error summary log to overflow

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c75089c332cd
[puppeteer] Skip test "$$eval should handle many elements" which is causing the error summary log to overflow r=webdriver-reviewers,jdescottes

Comment 8

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/c75089c332cd