808104 - Speculatively connect to the homepage before opening the first window

Status: NEW

(Firefox :: General, defect, P3)

Description

[Dão Gottwald [:dao]]

Speculatively connect to the homepage (in case it's a remote one) before opening the first window.

Comment 1

[Evelyn Hung]

I spent some time to read the history from bug 756313 and realized that we moved firstLoadURI after MozAfterPaint to speed up Chrome Window's startup time. Although this bug doesn't have detailed description but I think we want to setup speculative connection *before* the MozAfterPaint since it won't block main thread's startup jobs, so we can still take advantage of the time for speeding up content loading.

I am working on a WIP patch and will upload it tomorrow.

Comment 2

[Evelyn Hung]

Attachment: Bug 808104 - try to speculatively setup connection for homepage.

We want the connection being setup as soon as possible when we know
what uri will be opened.

Review commit: https://reviewboard.mozilla.org/r/145640/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/145640/

Comment 3

[Evelyn Hung]

I put the speculatively connection setup in the |openwindow()| of nsBrowserContentHandler.js, which (I believe) is the very beginning on the startup path that we know the homepage url (or url from the command line). I also tried |onLoad()| of browser.js where we have xul window ready but right before it painted, but then I felt we can do it earlier.

I also wonder what principal should be applied. I guess that the origin attribute check doesn't matter in our case so I pass an empty dictionary `{}` for |createCodebasePrincipal()|.

Another small issue I bumped into was that I couldn't log any activity of this connection setup with profiler markers. I added a marker in my |maybeSetupSpeculativeConnection()| and another one in |nsIOService::SpeculativeConnectInternal()|, and none of them showed on my profile result. I'm sure the |maybeSetupSpeculativeConnection()| is called (by dump console log), so maybe it happened to be too early so that even |MOZ_PROFILER_STARTUP=1 ./mach run| couldn't catch them? Note that I see my markers if the code lives in |onLoad()| of browser.js.

:mconley, I know you are one of the experts of Firefox start-up. Could you check out my patch and feedback? Thank you. :)

Comment 4

[Florian Quèze [:florian]]

I wonder what that means for NSS initialization. We are pretty close to ensuring NSS isn't initialized on the main thread before first paint, but ideally we would want NSS initialized off-main thread with a promise letting us know once it's OK to use the network.

Comment 5

[Evelyn Hung]

(In reply to Evelyn Hung [:evelyn] from comment #3) 
> Another small issue I bumped into was that I couldn't log any activity of
> this connection setup with profiler markers. I added a marker in my
> |maybeSetupSpeculativeConnection()| and another one in
> |nsIOService::SpeculativeConnectInternal()|, and none of them showed on my
> profile result. I'm sure the |maybeSetupSpeculativeConnection()| is called
> (by dump console log), so maybe it happened to be too early so that even
> |MOZ_PROFILER_STARTUP=1 ./mach run| couldn't catch them? Note that I see my
> markers if the code lives in |onLoad()| of browser.js.

On an optimized build, I now can see my profile markers (https://perfht.ml/2rshuds). Sorry I was on a debug build, maybe that's why it looked weird.

(In reply to Florian Quèze [:florian] [:flo] from comment #4)
> I wonder what that means for NSS initialization. We are pretty close to
> ensuring NSS isn't initialized on the main thread before first paint, but
> ideally we would want NSS initialized off-main thread with a promise letting
> us know once it's OK to use the network.

Hmmm... I don't know. Will we queue the speculative connection request for its initialization?

Hi Patrick, how do we know the network initialization is done so we can request for a speculative connection at Firefox _startup_? Also, specific to Florian's question, if NSS initialization isn't ready, will we be able to setup a speculative connection with ssl?

Comment 6

[Mike Conley (:mconley) (:⚙️)]

Tentatively, this looks okay, but I'll note that with e10s enabled, about:home will be loaded in the content process. Do we know if speculative connection / socket-warmup in the parent process has any impact on loads that occur in content processes? It would seem to make sense for them to, since (I believe) networking ultimately goes through the parent process... but I'd like to make sure before we really start to lean in on this speculative connection stuff.

Comment 7

[Patrick McManus [:mcmanus]]

nss will be loaded on demand

Comment 8

[Evelyn Hung]

(In reply to Mike Conley (:mconley) from comment #6)
> Tentatively, this looks okay, but I'll note that with e10s enabled,
> about:home will be loaded in the content process. Do we know if speculative
> connection / socket-warmup in the parent process has any impact on loads
> that occur in content processes? It would seem to make sense for them to,
> since (I believe) networking ultimately goes through the parent process...
> but I'd like to make sure before we really start to lean in on this
> speculative connection stuff.

Yes, I asked the question to :schien (network team in Taipei) and the answer was the pool of network connection are shared among processes, and we only separate connections for private browsing. 

I tried to find a place in network module to print out how my speculative connection is used for the homepage loading, so we can verify the logic is correct and things happen as we expected, i.e. the connection is ready before we start fetching the page. However, I found it's probably hard to know if the http request is using the speculative connection.

:dragana, the case here is that I setup a speculative connection for a URL, and I would like to know if the connection is used for my later http request to that URL. Is there a way to verify that?

Another question is, I request the speculative connection in Firefox _startup_ time at the point that xul window isn't created yet. What principal and origin attributes should I pass into the speculative connection so that it can be used later by a tab's docshell for loading the URL?

Comment 9

[Florian Quèze [:florian]]

(In reply to Evelyn Hung [:evelyn] from comment #5)

> (In reply to Florian Quèze [:florian] [:flo] from comment #4)
> > I wonder what that means for NSS initialization. We are pretty close to
> > ensuring NSS isn't initialized on the main thread before first paint, but
> > ideally we would want NSS initialized off-main thread with a promise letting
> > us know once it's OK to use the network.

I filed bug 1370516 on this.

> if NSS initialization isn't ready, will we be able to
> setup a speculative connection with ssl?

No, an ssl connection needs NSS to be initialized. If we cause NSS to be initialized synchronously on the main thread, it'll be a bigger regression than the win we are hoping for with the speculative connection.

Comment 10

[Evelyn Hung]

(In reply to Florian Quèze [:florian] [:flo] from comment #9) 
> No, an ssl connection needs NSS to be initialized. If we cause NSS to be
> initialized synchronously on the main thread, it'll be a bigger regression
> than the win we are hoping for with the speculative connection.

Yeah, that's what I learned. As Patrick said in comment 7, the initialization is loaded on demand. Therefore, being the first one to trigger it on this critical path sounds not a good idea, but I wonder how long will the initialization take. I will profile how much time has been spent on my |maybeSetupSpeculativeConnection|. If it does take long time, that means we should avoid networking before MozAfterPaint, which means there is nothing we can do here, right?

Comment 11

[Evelyn Hung]

(In reply to Evelyn Hung [:evelyn] from comment #10) 
>If it does take long time,
> that means we should avoid networking before MozAfterPaint, which means
> there is nothing we can do here, right?

Oh, I just read bug 1370516, let see if it will get fixed in 57.

Comment 12

[Florian Quèze [:florian]]

(In reply to Evelyn Hung [:evelyn] from comment #10)
> If it does take long time,
> that means we should avoid networking before MozAfterPaint, which means
> there is nothing we can do here, right?

I'm afraid there's nothing we can do until bug 1370516 is fixed, yes. While we wait for an answer/a fix there, I would suggest working on another speculative connection bug, eg. bug 1348275.

Comment 13

[Dragana Damjanovic [:dragana]]

(In reply to Evelyn Hung [:evelyn] from comment #8)
> (In reply to Mike Conley (:mconley) from comment #6)
> > Tentatively, this looks okay, but I'll note that with e10s enabled,
> > about:home will be loaded in the content process. Do we know if speculative
> > connection / socket-warmup in the parent process has any impact on loads
> > that occur in content processes? It would seem to make sense for them to,
> > since (I believe) networking ultimately goes through the parent process...
> > but I'd like to make sure before we really start to lean in on this
> > speculative connection stuff.
> 
> Yes, I asked the question to :schien (network team in Taipei) and the answer
> was the pool of network connection are shared among processes, and we only
> separate connections for private browsing. 
> 
> I tried to find a place in network module to print out how my speculative
> connection is used for the homepage loading, so we can verify the logic is
> correct and things happen as we expected, i.e. the connection is ready
> before we start fetching the page. However, I found it's probably hard to
> know if the http request is using the speculative connection.
> 
> :dragana, the case here is that I setup a speculative connection for a URL,
> and I would like to know if the connection is used for my later http request
> to that URL. Is there a way to verify that?
> 

There is no easy way. You can look at logs. From logs you can see if your transaction uses speculative connection.

> Another question is, I request the speculative connection in Firefox
> _startup_ time at the point that xul window isn't created yet. What
> principal and origin attributes should I pass into the speculative
> connection so that it can be used later by a tab's docshell for loading the
> URL?

:ckreschb can answer about principal.

Comment 14

[Christoph Kerschbaumer [:ckerschb]]

Comment on attachment 8874240 [details]
Bug 808104 - try to speculatively setup connection for homepage.

https://reviewboard.mozilla.org/r/145640/#review150618

::: browser/components/nsBrowserContentHandler.js:90
(Diff revision 1)
> +  }
> +  let sc = Services.io.QueryInterface(Components.interfaces.nsISpeculativeConnect);
> +  try {
> +    let uri = Services.io.newURI(uristring);
> +    let principal = Services.scriptSecurityManager
> +                    .createCodebasePrincipal(uri, {});

Please *do not* create a principal using the URI that is about to be loaded. That basically bypasses all security checks. If the URI is coming from the web, an attacker might use that functionality to exfiltrate information.

What we should use as the principal is the page that contains the uri to be loaded. I guess you have to pass that principal around so you can use it here.

Comment 15

[Christoph Kerschbaumer [:ckerschb]]

See my previous comment.

Comment 16

[Evelyn Hung]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #14) 
> Please *do not* create a principal using the URI that is about to be loaded.
> That basically bypasses all security checks. If the URI is coming from the
> web, an attacker might use that functionality to exfiltrate information.
> 
> What we should use as the principal is the page that contains the uri to be
> loaded. I guess you have to pass that principal around so you can use it
> here.

Thanks for the input. The URI is from the user who setup a homepage url on his/her Firefox preference panel.

Comment 17

[Florian Quèze [:florian]]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #14)

> Please *do not* create a principal using the URI that is about to be loaded.
> That basically bypasses all security checks. If the URI is coming from the
> web, an attacker might use that functionality to exfiltrate information.
> 
> What we should use as the principal is the page that contains the uri to be
> loaded. I guess you have to pass that principal around so you can use it
> here.

I think you missed the point of Evelyn's question. There's no "page that contains the uri to be loaded", there's not even a browser window yet. This is during the browser's startup, and the url is coming from the user's preferences, it's the initial load of the homepage.

Comment 18

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Florian Quèze [:florian] [:flo] from comment #17)
> I think you missed the point of Evelyn's question. There's no "page that
> contains the uri to be loaded", there's not even a browser window yet. This
> is during the browser's startup, and the url is coming from the user's
> preferences, it's the initial load of the homepage.

I don't think I have, but let's make sure. The question was:

> What principal and origin attributes should I pass into the speculative
> connection so that it can be used later by a tab's docshell for loading the
> URL?

In that case we should *not* create a principal using the URI that is about to be loaded. At this point there is also no intervention from the user, right? The user has not clicked on a link yet, right? In that case I think we should use the principal of the page that basically *will* cause a navigation for the page. Similar to a frame navigation. I agree that the page's principal is not ultimately right in the end. But following the principle of least privilege, this is the principal we should use to speculatively connect.

Comment 19

[Christoph Kerschbaumer [:ckerschb]]

Oh sorry, it's really happening at startup and only at startup? If that is the case and the URL to be loaded can't be influenced by any webpage, but only by the user's settings, then probably creating a principal using the URI to be loaded is fine. Sorry for the confusion.

Comment 20

[Evelyn Hung]

Thanks for helping me clarify this, :cherschb and Florian. Good to know I did it right.

Comment 21

[Mike Conley (:mconley) (:⚙️)]

Hey evelyn! Are you still working on this?

Comment 22

[Evelyn Hung]

No, I'm hoping someone to take it over. Let me deassign myself. Thanks for reminding me.

Comment 23

[Florian Quèze [:florian]]

I would be tempted to make this a P2, but it's tricky until NSS is initialized asynchronously (bug 
1370516). Currently, starting a load of any https URL early would significantly delay first paint, and be a regression rather than a perf improvement.