1771867 - Early hint preloader, phase 2, informing the speculative loader

Status: RESOLVED FIXED

(Core :: Networking: HTTP, task, P2)

Description

[Manuel Bucher [:manuel]]

Follow up on 1753730#c12 to make the preloader reliable.

Might help with Devtools (Bug 1761248).

Comment 1

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 12: Redirect EarlyHint preload channel to content process r=#necko

Comment 2

[Manuel Bucher [:manuel]]

Attachment: WIP: Bug 1771867 - Make EarlyHintPreloader accessible to the content process via IPC

Depends on D149643

Comment 3

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 13: Make early hint tests strict r=#necko

Depends on D149643

Comment 4

[Manuel Bucher [:manuel]]

Attachment: WIP: Bug 1771867 - Redirect EarlyHintPreloader to content process similar to how DocumentLoadListener does it

Depends on D154222

Comment 5

[Manuel Bucher [:manuel]]

Attachment: WIP: Bug 1771867 - Phase 2 Early Hint Preloader, Part 2: Connect Preloader back to EarlyHintPreloader

Depends on D154222

Comment 6

[Manuel Bucher [:manuel]]

Attachment: WIP: Bug 1771867 - Phase 2 preloader, part 2: connect Preloader back to EarlyHintPreloader

Depends on D154222

Comment 7

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Move structs to NeckoCommon to reuse them in Early Hint Preloader code r=#necko

Comment 8

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Move structs to NeckoCommon to reuse them in Early Hint Preloader code r=#necko

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8bd3e0b3d155
Move structs to NeckoCommon to reuse them in Early Hint Preloader code r=necko-reviewers,valentin

Comment 10

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/8bd3e0b3d155

Comment 11

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 1: Add ongoing early hint preloads to http IPC channel args r=#necko

Comment 12

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 2: Pass early hint preloads to preloader from IPC r=#necko

Depends on D161172

Comment 13

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 4: Pass early hint preload to fetch and font preloader r=#necko

Depends on D161173

Comment 14

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 5: Pass early hint preload to script preloader r=#necko

Note that modules can't be specified in Link preloads with rel=preload,
only in rel=modulepreload. We currently only support rel=preload in
early hints. See Bug 1798319 for updates on module preloads.

Depends on D161174

Comment 15

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 6: Pass early hint preload to style preloader r=#necko

Depends on D161175

Comment 16

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 7: Pass early hint preload to image preloader r=#necko

Depends on D161176

Comment 17

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 3: Add mechanism to notify EarlyHintPreloader that HttpChannelParent is ready to complete the redirect r=#necko

Depends on D161177

Comment 18

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 8: Add list of open channels from OngoingEarlyHints to RedirectToRealChannelArgs r=#necko

Depends on D161178

Comment 19

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 9: Don't cancel redirects to insecure http early hint preloads r=#necko

Previously those got cancelled to discourage http usage in new components.
Preloading them now to make this preloader behave the same way as the
preloader in the content process.

Depends on D161179

Comment 20

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 10: Only use first early hint reponse r=#necko

to make implementation more spec conform

Depends on D161180

Comment 21

[Manuel Bucher [:manuel]]

Attachment: Bug 1771867 - Early Hints Phase 2 - Part 11: Don't preload anchored links from early hints r=#necko

They won't get passed to the PreloadService anyway. Resulting in an unnecessary
additional load

Depends on D161181

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f8a03d226c73
Early Hints Phase 2 - Part 1: Add ongoing early hint preloads to http IPC channel args r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/5090eae24a5c
Early Hints Phase 2 - Part 2: Pass early hint preloads to preloader from IPC r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/7cdf3cef2773
Early Hints Phase 2 - Part 3: Add mechanism to notify EarlyHintPreloader that HttpChannelParent is ready to complete the redirect r=necko-reviewers,kershaw,valentin
https://hg.mozilla.org/integration/autoland/rev/b3d65b1aa872
Early Hints Phase 2 - Part 4: Pass early hint preload to fetch and font preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/7f20525f5e94
Early Hints Phase 2 - Part 5: Pass early hint preload to script preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/110ac12e16f5
Early Hints Phase 2 - Part 6: Pass early hint preload to style preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/d33ccbbf407d
Early Hints Phase 2 - Part 7: Pass early hint preload to image preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/1177913e1edf
Early Hints Phase 2 - Part 8: Add list of open channels from OngoingEarlyHints to RedirectToRealChannelArgs r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/a9f3158ed688
Early Hints Phase 2 - Part 9: Don't cancel redirects to insecure http early hint preloads r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/8bdb1f682d22
Early Hints Phase 2 - Part 10: Only use first early hint reponse r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/2fe2afdee09d
Early Hints Phase 2 - Part 11: Don't preload anchored links from early hints r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/aea7ca0bbd17
Early Hints Phase 2 - Part 12: Redirect EarlyHint preload channel to content process r=necko-reviewers,kershaw,valentin
https://hg.mozilla.org/integration/autoland/rev/120838b58449
Early Hints Phase 2 - Part 13: Make early hint tests strict r=necko-reviewers,kershaw,valentin

Comment 23

[Natalia Csoregi [:nataliaCs]]

Backed out for causing CanonicalBrowsingContext crashes.

• backout: https://hg.mozilla.org/integration/autoland/rev/e2b41473c7ed7b21209ff83fd80017a4f7b5de8f
• push: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=L7wuCkh5TW207Ey7maeE6Q.0&revision=120838b58449ab57a04eee2823ead4054efd2e05
• failure log: https://treeherder.mozilla.org/logviewer?job_id=398438231&repo=autoland&lineNumber=5025

[task 2022-12-02T11:05:19.692Z] 11:05:19  WARNING -  PROCESS-CRASH | image/test/mochitest/test_bug468160.html | application crashed [@ RefPtr<mozilla::dom::CanonicalBrowsingContext>::operator->() const]

• another failure log: PROCESS-CRASH | dom/security/test/https-only/test_resource_upgrade.html | application crashed [@ RefPtr<mozilla::dom::CanonicalBrowsingContext>::operator->() const]

Comment 24

[Manuel Bucher [:manuel]]

Will figure out a fix (and run more tests next time). backtrace:

(gdb) bt
#0  RefPtr<mozilla::dom::CanonicalBrowsingContext>::operator->() const (this=<optimized out>) at /home/user/dev/gecko/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h:314
#1  0x00007fffeab93071 in mozilla::net::DocumentLoadListener::AddURIVisit(nsIChannel*, unsigned int) (this=this@entry=0x7fffbfe4b8d0, aChannel=aChannel@entry=0x7fffac026250, aLoadFlags=<optimized out>)
    at /home/user/dev/gecko/netwerk/ipc/DocumentLoadListener.cpp:448
#2  0x00007fffeab9bb66 in mozilla::net::DocumentLoadListener::AsyncOnChannelRedirect(nsIChannel*, nsIChannel*, unsigned int, nsIAsyncVerifyRedirectCallback*)
    (this=0x7fffbfe4b8d0, aOldChannel=0x7fffac026250, aNewChannel=0x7fffac026c50, aFlags=1, aCallback=0x7fffa99212f0) at /home/user/dev/gecko/netwerk/ipc/DocumentLoadListener.cpp:2775
#3  0x00007fffeab9bd97 in non-virtual thunk to mozilla::net::DocumentLoadListener::AsyncOnChannelRedirect(nsIChannel*, nsIChannel*, unsigned int, nsIAsyncVerifyRedirectCallback*) ()
    at /home/user/dev/gecko/obj-x86_64-pc-linux-gnu/dist/bin/libxul.so
#4  0x00007fffec937f6d in mozilla::dom::(anonymous namespace)::ClientChannelHelper::AsyncOnChannelRedirect(nsIChannel*, nsIChannel*, unsigned int, nsIAsyncVerifyRedirectCallback*)
    (this=0x7fffbfeb1780, aOldChannel=0x7fffac026250, aNewChannel=0x7fffac026c50, aFlags=1, aCallback=0x7fffa99212f0) at /home/user/dev/gecko/dom/clients/manager/ClientChannelHelper.cpp:174
#5  0x00007fffea64b397 in mozilla::net::nsAsyncRedirectVerifyHelper::DelegateOnChannelRedirect(nsIChannelEventSink*, nsIChannel*, nsIChannel*, unsigned int)
    (this=this@entry=0x7fffa99212e0, sink=0x7fffbfeb1788, oldChannel=0x7fffac026250, newChannel=0x7fffac026c50, flags=1) at /home/user/dev/gecko/netwerk/base/nsAsyncRedirectVerifyHelper.cpp:155
#6  0x00007fffea64b662 in mozilla::net::nsAsyncRedirectVerifyHelper::Run() (this=0x7fffa99212e0) at /home/user/dev/gecko/netwerk/base/nsAsyncRedirectVerifyHelper.cpp:260
#7  0x00007fffea4d2fe5 in mozilla::RunnableTask::Run() (this=0x7fffbd94fa00) at /home/user/dev/gecko/xpcom/threads/TaskController.cpp:538
#8  0x00007fffea4cc751 in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) (this=this@entry=0x7fffe2622d00, aProofOfLock=...)
    at /home/user/dev/gecko/xpcom/threads/TaskController.cpp:851
#9  0x00007fffea4cb97e in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) (this=this@entry=0x7fffe2622d00, aProofOfLock=...)
    at /home/user/dev/gecko/xpcom/threads/TaskController.cpp:683
#10 0x00007fffea4cbbcb in mozilla::TaskController::ProcessPendingMTTask(bool) (this=0x7fffe2622d00, aMayWait=false) at /home/user/dev/gecko/xpcom/threads/TaskController.cpp:461
#11 0x00007fffea4ccb7d in mozilla::TaskController::InitializeInternal()::$_2::operator()() const (this=<optimized out>) at /home/user/dev/gecko/xpcom/threads/TaskController.cpp:187
#12 mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_2>::Run() (this=<optimized out>) at /home/user/dev/gecko/obj-x86_64-pc-linux-gnu/dist/include/nsThreadUtils.h:546
#13 0x00007fffea4e3647 in nsThread::ProcessNextEvent(bool, bool*) (this=0x7ffff78a3600, aMayWait=<optimized out>, aResult=0x7fffffffc4d7) at /home/user/dev/gecko/xpcom/threads/nsThread.cpp:1204
#14 0x00007fffea4e7211 in NS_ProcessNextEvent(nsIThread*, bool) (aThread=0x7ffff7bf3940, aThread@entry=0x7ffff78a3600, aMayWait=false) at /home/user/dev/gecko/xpcom/threads/nsThreadUtils.cpp:474
#15 0x00007fffead2db46 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) (this=0x7fffe273bdc0, aDelegate=0x7ffff7825c40) at /home/user/dev/gecko/ipc/glue/MessagePump.cpp:85
#16 0x00007fffeaca2a3f in MessageLoop::RunInternal() (this=this@entry=0x7ffff7825c40) at /home/user/dev/gecko/ipc/chromium/src/base/message_loop.cc:381
#17 0x00007fffeaca298f in MessageLoop::RunHandler() (this=0x7ffff7825c40) at /home/user/dev/gecko/ipc/chromium/src/base/message_loop.cc:374
#18 MessageLoop::Run() (this=0x7ffff7825c40) at /home/user/dev/gecko/ipc/chromium/src/base/message_loop.cc:356
#19 0x00007fffedad1bf7 in nsBaseAppShell::Run() (this=0x7fffe2778900) at /home/user/dev/gecko/widget/nsBaseAppShell.cpp:150
#20 0x00007fffef29061c in nsAppStartup::Run() (this=0x7fffe2724fb0) at /home/user/dev/gecko/toolkit/components/startup/nsAppStartup.cpp:295
#21 0x00007fffef3a1385 in XREMain::XRE_mainRun() (this=this@entry=0x7fffffffc850) at /home/user/dev/gecko/toolkit/xre/nsAppRunner.cpp:5767
#22 0x00007fffef3a1ec7 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) (this=this@entry=0x7fffffffc850, argc=argc@entry=5, argv=argv@entry=0x7fffffffdb58, aConfig=...)
    at /home/user/dev/gecko/toolkit/xre/nsAppRunner.cpp:5960
#23 0x00007fffef3a2333 in XRE_main(int, char**, mozilla::BootstrapConfig const&) (argc=5, argv=0x7fffffffdb58, aConfig=...) at /home/user/dev/gecko/toolkit/xre/nsAppRunner.cpp:6016
#24 0x000055555559fdf6 in do_main(int, char**, char**) (argc=5, argv=0x7fffffffdb58, envp=0x7fffffffdb88) at /home/user/dev/gecko/browser/app/nsBrowserApp.cpp:226
#25 main(int, char**, char**) (argc=<optimized out>, argv=<optimized out>, envp=0x7fffffffdb88) at /home/user/dev/gecko/browser/app/nsBrowserApp.cpp:430

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7a267d40f4ef
Early Hints Phase 2 - Part 1: Add ongoing early hint preloads to http IPC channel args r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/13a9f4631369
Early Hints Phase 2 - Part 2: Pass early hint preloads to preloader from IPC r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/b4b920f24bed
Early Hints Phase 2 - Part 3: Add mechanism to notify EarlyHintPreloader that HttpChannelParent is ready to complete the redirect r=necko-reviewers,kershaw,valentin
https://hg.mozilla.org/integration/autoland/rev/c98f651b7439
Early Hints Phase 2 - Part 4: Pass early hint preload to fetch and font preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/ae5aec815b28
Early Hints Phase 2 - Part 5: Pass early hint preload to script preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/b348a6911423
Early Hints Phase 2 - Part 6: Pass early hint preload to style preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/ce8f0a8fa688
Early Hints Phase 2 - Part 7: Pass early hint preload to image preloader r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/8575cd92e1d8
Early Hints Phase 2 - Part 8: Add list of open channels from OngoingEarlyHints to RedirectToRealChannelArgs r=necko-reviewers,valentin,kershaw
https://hg.mozilla.org/integration/autoland/rev/a6a5172b9746
Early Hints Phase 2 - Part 9: Don't cancel redirects to insecure http early hint preloads r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/cbbdb4b4aaa9
Early Hints Phase 2 - Part 10: Only use first early hint reponse r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/ca2681b0f792
Early Hints Phase 2 - Part 11: Don't preload anchored links from early hints r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/35dde62c945a
Early Hints Phase 2 - Part 12: Redirect EarlyHint preload channel to content process r=necko-reviewers,kershaw,valentin
https://hg.mozilla.org/integration/autoland/rev/f9edacd5c749
Early Hints Phase 2 - Part 13: Make early hint tests strict r=necko-reviewers,kershaw,valentin

Comment 26

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/7a267d40f4ef
https://hg.mozilla.org/mozilla-central/rev/13a9f4631369
https://hg.mozilla.org/mozilla-central/rev/b4b920f24bed
https://hg.mozilla.org/mozilla-central/rev/c98f651b7439
https://hg.mozilla.org/mozilla-central/rev/ae5aec815b28
https://hg.mozilla.org/mozilla-central/rev/b348a6911423
https://hg.mozilla.org/mozilla-central/rev/ce8f0a8fa688
https://hg.mozilla.org/mozilla-central/rev/8575cd92e1d8
https://hg.mozilla.org/mozilla-central/rev/a6a5172b9746
https://hg.mozilla.org/mozilla-central/rev/cbbdb4b4aaa9
https://hg.mozilla.org/mozilla-central/rev/ca2681b0f792
https://hg.mozilla.org/mozilla-central/rev/35dde62c945a
https://hg.mozilla.org/mozilla-central/rev/f9edacd5c749

Comment 27

[Phabricator Automation]

Comment on attachment 9294944 [details]
Bug 1771867 - Move structs to NeckoCommon to reuse them in Early Hint Preloader code r=#necko

Revision D136612 was moved to bug 1733498. Setting attachment 9294944 [details] to obsolete.