Page.navigate never resolves on some URLs
Categories
(Remote Protocol :: CDP, defect, P3)
Tracking
(firefox70 fixed)
| Tracking | Status | |
|---|---|---|
| firefox70 | --- | fixed |
People
(Reporter: jdescottes, Assigned: jdescottes)
References
Details
Attachments
(1 file)
On a few URLs, using puppeteer's page.goto (which relies on Page.navigate) never resolves.
For instance, the following basic puppeteer test never finishes:
console.log("Create page");
const page = await browser.newPage();
console.log("Go to https://www.lemonde.fr/");
await page.goto("https://www.lemonde.fr/");
console.log("Wait for 1 seconds");
await new Promise(r => setTimeout(r, 1000));
console.log("Close the browser");
await browser.close();
|
||
Comment 1•5 years ago
|
||
Could it be because pageshow is not firing?
I know that for DevTools we can't rely on load event because, for example, the event can be fired very very late because of ads iframes. Or just never fires if document.open and/or document.write is used.
So that may be similar, but with pageshow?
The workaround for load event was to use the nsIProgressListener over here:
https://searchfox.org/mozilla-central/source/devtools/server/actors/targets/browsing-context.js#1589-1593
https://searchfox.org/mozilla-central/source/devtools/server/actors/targets/browsing-context.js#1717
|
Assignee | |
Comment 2•5 years ago
|
||
It seems related to iframes in general, not necessarily slow ones. The simple test page below also triggers the issue.
<!DOCTYPE html>
<html lang="en">
<body>
<iframe src="http://google.com"></iframe>
</body>
</html>
There might be several problems, but one of the issues here is that we emit frameNavigated for the iframe's frame as if it was a top level frame.
onFrameNavigated(name, { frameId, window }) {
const url = window.location.href;
this.emit("Page.frameNavigated", {
frame: {
id: frameId,
// frameNavigated is only emitted for the top level document
// so that it never has a parent.
parentId: null,
url,
},
});
}
remote/domains/content/Page.jsm#108
Consequently, the iframe's frame becomes the main frame from puppeteers point of view. See FrameManager::onFrameNavigated https://github.com/GoogleChrome/puppeteer/blob/v1.6.2/lib/FrameManager.js#L141-L152
Later on Firefox will receive "pageshow" and fire "frameStoppedLoading", but for the main frame only. So puppeteer never receives frameStoppedLoading for the iframe and thinks the page never finished loading.
Comparing more what Firefox and Chrome do, I think that to support fully iframes we need more work. Apparently, Chrome notifies puppeteer of iframes via Page.frameAttached, which we track in Bug 1549512.
But I think we should update onFrameNavigated to verify that the "frame-navigated" event we received was from the top level window, and not from an iframe.
|
|
Assignee | |
Comment 3•5 years ago
|
||
WIP to illustrate a fix for this issue
|
|
Assignee | |
Updated•5 years ago
|
Pushed by jdescottes@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4b05fdad2e13 Ignore iframes for Page.frameNavigated r=remote-protocol-reviewers,ochameau
|
||
Comment 5•5 years ago
|
||
| bugherder | ||
|
||
Comment 6•5 years ago
|
||
I spotted this too late, but this is some great detective work!
|
||
Updated•5 years ago
|
|
||
Updated•3 years ago
|
Description
•