1860603 - Perma wd BETA TEST-UNEXPECTED-PASS | OOPIF-debug OOPIF should support lazy OOP frames (oopif.spec.js) | expected FAIL

Status: RESOLVED FIXED

(Remote Protocol :: Agent, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=433434221&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bhWclCHfQZq127QegBNoIA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-10-23T16:04:33.224Z] PID 693 | 1698077073223	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 <- {"type":"event","method":"browsingContext.load","params":{"context":"91085674-52f4-4d4c-9994-bb4d496195ff","timestamp":1698077073219,"url":"http://localhost:33809/lazy-oopif-frame.html","navigation":"a3f8309d-e6ff-4e84-a163-9b3b5575ab9c"}}
[task 2023-10-23T16:04:33.229Z] PID 693 | 1698077073228	RemoteAgent	TRACE	[14] ProgressListener Check loading state: isStart=0 isStop=16
[task 2023-10-23T16:04:33.229Z] PID 693 | 1698077073229	RemoteAgent	TRACE	[14] ProgressListener state=stop: http://localhost:33809/lazy-oopif-frame.html
[task 2023-10-23T16:04:33.229Z] PID 693 | 1698077073229	RemoteAgent	TRACE	[14] ProgressListener Stop: has error=false
[task 2023-10-23T16:04:33.232Z] PID 693 | 1698077073231	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 <- {"type":"success","id":8,"result":{"navigation":"a3f8309d-e6ff-4e84-a163-9b3b5575ab9c","url":"http://localhost:33809/lazy-oopif-frame.html"}}
[task 2023-10-23T16:04:33.234Z] PID 693 | 1698077073233	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 <- {"type":"event","method":"network.beforeRequestSent","params":{"context":"91085674-52f4-4d4c-9994-bb4d496195ff","isBlocked":false,"navigation":null,"redirectCount":0,"request":{"bodySize":null,"cookies":[],"headers":[{"name":"Host","value":{"type":"string","value":"localhost:33809"}},{"name":"User-Agent","value":{"type":"string","value":"Mozilla/5.0 (X11; Linux x86_64; rv:120.0) Gecko/20100101 Firefox/120.0"}},{"name":"Accept","value":{"type":"string","value":"image/avif,image/webp,*/*"}},{"name":"Accept-Language","value":{"type":"string","value":"en-US,en;q=0.5"}},{"name":"Accept-Encoding","value":{"type":"string","value":"gzip, deflate, br"}},{"name":"Referer","value":{"type":"string","value":"http://localhost:33809/lazy-oopif-frame.html"}},{"name":"Connection","value":{"type":"string","value":"keep-alive"}},{"name":"Sec-Fetch-Dest","value":{"type":"string","value":"image"}},{"name":"Sec-Fetch-Mode","value":{"type":"string","value":"no-cors"}},{"name":"Sec-Fetch-Site","value":{"type":"string","value":"same-origin"}}],"headersSize":396,"method":"GET","request":"35","timings":{"timeOrigin":0,"requestTime":1698077073201412,"redirectStart":0,"redirectEnd":0,"fetchStart":0,"dnsStart":0,"dnsEnd":0,"connectStart":0,"connectEnd":0,"tlsStart":0,"tlsEnd":0,"requestStart":1698077073231528,"responseStart":0,"responseEnd":0},"url":"http://localhost:33809/favicon.ico"},"timestamp":1698077073233,"initiator":{"type":"other"}}}
[task 2023-10-23T16:04:33.235Z] PID 693 | 1698077073234	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 -> {"id":9,"method":"browsingContext.setViewport","params":{"context":"91085674-52f4-4d4c-9994-bb4d496195ff","viewport":{"width":1000,"height":1000}}}
[task 2023-10-23T16:04:33.235Z] PID 693 | 1698077073234	RemoteAgent	TRACE	Module root/browsingContext.sys.mjs found for ROOT
[task 2023-10-23T16:04:33.235Z] PID 693 | 1698077073234	RemoteAgent	TRACE	Received command browsingContext.setViewport for destination ROOT
[task 2023-10-23T16:04:33.235Z] PID 693 | 1698077073234	RemoteAgent	TRACE	Module root/browsingContext.sys.mjs found for ROOT
[task 2023-10-23T16:04:33.236Z] PID 693 | 1698077073236	RemoteAgent	TRACE	Received command browsingContext._awaitViewportDimensions for destination WINDOW_GLOBAL
[task 2023-10-23T16:04:33.237Z] PID 693 | 1698077073236	RemoteAgent	TRACE	Module windowglobal/browsingContext.sys.mjs found for WINDOW_GLOBAL
[task 2023-10-23T16:04:33.238Z] PID 693 | 1698077073238	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 <- {"type":"event","method":"network.responseStarted","params":{"context":"91085674-52f4-4d4c-9994-bb4d496195ff","isBlocked":false,"navigation":null,"redirectCount":0,"request":{"bodySize":null,"cookies":[],"headers":[{"name":"Host","value":{"type":"string","value":"localhost:33809"}},{"name":"User-Agent","value":{"type":"string","value":"Mozilla/5.0 (X11; Linux x86_64; rv:120.0) Gecko/20100101 Firefox/120.0"}},{"name":"Accept","value":{"type":"string","value":"image/avif,image/webp,*/*"}},{"name":"Accept-Language","value":{"type":"string","value":"en-US,en;q=0.5"}},{"name":"Accept-Encoding","value":{"type":"string","value":"gzip, deflate, br"}},{"name":"Referer","value":{"type":"string","value":"http://localhost:33809/lazy-oopif-frame.html"}},{"name":"Connection","value":{"type":"string","value":"keep-alive"}},{"name":"Sec-Fetch-Dest","value":{"type":"string","value":"image"}},{"name":"Sec-Fetch-Mode","value":{"type":"string","value":"no-cors"}},{"name":"Sec-Fetch-Site","value":{"type":"string","value":"same-origin"}}],"headersSize":396,"method":"GET","request":"35","timings":{"timeOrigin":0,"requestTime":1698077073201412,"redirectStart":0,"redirectEnd":0,"fetchStart":0,"dnsStart":0,"dnsEnd":0,"connectStart":0,"connectEnd":0,"tlsStart":0,"tlsEnd":0,"requestStart":1698077073231528,"responseStart":1698077073233421,"responseEnd":1698077073233492},"url":"http://localhost:33809/favicon.ico"},"timestamp":1698077073237,"response":{"bodySize":0,"content":{"size":null},"headersSize":198,"url":"http://localhost:33809/favicon.ico","bytesReceived":198,"fromCache":false,"headers":[{"name":"Cache-Control","value":{"type":"string","value":"no-cache, no-store"}},{"name":"Content-Type","value":{"type":"string","value":"image/vnd.microsoft.icon"}},{"name":"Date","value":{"type":"string","value":"Mon, 23 Oct 2023 16:04:33 GMT"}},{"name":"Connection","value":{"type":"string","value":"keep-alive"}},{"name":"Keep-Alive","value":{"type":"string","value":"timeout=5"}},{"name":"Content-Length","value":{"type":"string","value":"70"}}],"mimeType":"image/vnd.microsoft.icon","protocol":"http/1.1","status":200,"statusText":"OK"}}}
[task 2023-10-23T16:04:33.242Z] PID 693 | 1698077073241	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 <- {"type":"event","method":"network.responseCompleted","params":{"context":"91085674-52f4-4d4c-9994-bb4d496195ff","isBlocked":false,"navigation":null,"redirectCount":0,"request":{"bodySize":null,"cookies":[],"headers":[{"name":"Host","value":{"type":"string","value":"localhost:33809"}},{"name":"User-Agent","value":{"type":"string","value":"Mozilla/5.0 (X11; Linux x86_64; rv:120.0) Gecko/20100101 Firefox/120.0"}},{"name":"Accept","value":{"type":"string","value":"image/avif,image/webp,*/*"}},{"name":"Accept-Language","value":{"type":"string","value":"en-US,en;q=0.5"}},{"name":"Accept-Encoding","value":{"type":"string","value":"gzip, deflate, br"}},{"name":"Referer","value":{"type":"string","value":"http://localhost:33809/lazy-oopif-frame.html"}},{"name":"Connection","value":{"type":"string","value":"keep-alive"}},{"name":"Sec-Fetch-Dest","value":{"type":"string","value":"image"}},{"name":"Sec-Fetch-Mode","value":{"type":"string","value":"no-cors"}},{"name":"Sec-Fetch-Site","value":{"type":"string","value":"same-origin"}}],"headersSize":396,"method":"GET","request":"35","timings":{"timeOrigin":0,"requestTime":1698077073201412,"redirectStart":0,"redirectEnd":0,"fetchStart":0,"dnsStart":0,"dnsEnd":0,"connectStart":0,"connectEnd":0,"tlsStart":0,"tlsEnd":0,"requestStart":1698077073231528,"responseStart":1698077073233421,"responseEnd":1698077073233492},"url":"http://localhost:33809/favicon.ico"},"timestamp":1698077073241,"response":{"bodySize":70,"content":{"size":70},"headersSize":198,"url":"http://localhost:33809/favicon.ico","bytesReceived":268,"fromCache":false,"headers":[{"name":"Cache-Control","value":{"type":"string","value":"no-cache, no-store"}},{"name":"Content-Type","value":{"type":"string","value":"image/vnd.microsoft.icon"}},{"name":"Date","value":{"type":"string","value":"Mon, 23 Oct 2023 16:04:33 GMT"}},{"name":"Connection","value":{"type":"string","value":"keep-alive"}},{"name":"Keep-Alive","value":{"type":"string","value":"timeout=5"}},{"name":"Content-Length","value":{"type":"string","value":"70"}}],"mimeType":"image/vnd.microsoft.icon","protocol":"http/1.1","status":200,"statusText":"OK"}}}
[task 2023-10-23T16:04:33.259Z] PID 693 | 1698077073258	RemoteAgent	DEBUG	WebDriverBiDiConnection 8ecd4a17-c772-4e25-ba92-4d0d803b0981 <- {"type":"success","id":9,"result":{}}
[task 2023-10-23T16:04:33.276Z] TEST-UNEXPECTED-PASS | OOPIF-debug OOPIF should support lazy OOP frames (oopif.spec.js) | expected FAIL
[task 2023-10-23T16:04:33.276Z] TEST-INFO expected FAIL | took 277ms

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I wonder if we should just disable this feature for CDP for now given that we do not have a way to distinguish in the Puppeteer expectation data between Nightly and beta/release. Alternatively this test needs to be made PASS, FAIL, but then we should not forget to update it once again when the feature is enabled on all channels.

Comment 2

[Julian Descottes [:jdescottes]]

I would just uplift a patch to expect it to PASS on Beta, and keep it as FAIL on Nightly. Note that the test was completely wrong and was changed upstream after filing the issue at https://github.com/puppeteer/puppeteer/issues/11198

Comment 3

[Julian Descottes [:jdescottes]]

Actually let's make is FAIL, PASS on beta, just in case the preference gets flipped during the cycle (I don't think it should be the case but still).

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Julian Descottes [:jdescottes] from comment #2)

I would just uplift a patch to expect it to PASS on Beta, and keep it as FAIL on Nightly. Note that the test was completely wrong and was changed upstream after filing the issue at https://github.com/puppeteer/puppeteer/issues/11198

Note that we would end-up with the same situation with the next merge to beta in case of the feature is still not enabled on all channels by that time. Does the fix of the Puppeteer testcase actually make it stable for Nightly and Beta?

Comment 5

[Julian Descottes [:jdescottes]]

Attachment: Bug 1860603 - [remote] Update puppeteer oopif expectation to PASS, FAIL on beta

The lazy loading iframe feature was only enabled in Nightly, and we need to revert the expectation on Beta.

Comment 6

[Julian Descottes [:jdescottes]]

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #4)

(In reply to Julian Descottes [:jdescottes] from comment #2)

I would just uplift a patch to expect it to PASS on Beta, and keep it as FAIL on Nightly. Note that the test was completely wrong and was changed upstream after filing the issue at https://github.com/puppeteer/puppeteer/issues/11198

Note that we would end-up with the same situation with the next merge to beta in case of the feature is still not enabled on all channels by that time. Does the fix of the Puppeteer testcase actually make it stable for Nightly and Beta?

Yes, it makes it fail regardless of the preference.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1860057

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e1903ddbf2bb
[puppeteer] Synchronize the test page fix for oopif puppeteer tests r=webdriver-reviewers,whimboo

Comment 9

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/e1903ddbf2bb

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:jdescottes, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox120 to wontfix.

For more information, please visit BugBot documentation.

Comment 11

[Julian Descottes [:jdescottes]]

Comment on attachment 9359951 [details]
Bug 1860603 - [remote] Update puppeteer oopif expectation to PASS, FAIL on beta

Beta/Release Uplift Approval Request

• User impact if declined: perma failure on CI for beta
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): test only fix
• String changes made/needed:
• Is Android affected?: No

Comment 12

[Dianna Smith [:diannaS]]

Comment on attachment 9359951 [details]
Bug 1860603 - [remote] Update puppeteer oopif expectation to PASS, FAIL on beta

Approved for 120.0b3

Comment 13

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/1c4496030371

Comment 14

[Intermittent Failures Robot]

14 failures in 3657 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 14

Platform and build breakdown:

• linux1804-64: 14
  • opt: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1860603&startday=2023-10-23&endday=2023-10-29&tree=all