1246715 - Browser Toolbox without manual process management

Status: NEW

(DevTools :: Framework, defect, P3)

Description

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

The current Browser Toolbox runs its UI in separate process that connects back to the main one.

This has several UX downsides, like:

* A new process for the OS window manager (instead of just window / tab inside the main process)
* Use of remote connections means an auth prompt on each connection

If we could change to something either in process or content process, that could alleviate these issues.

Comment 1

[Alexandre Poirot [:ochameau]]

Bug 1233463 is going to open ways to do that.

My current patch only supports targetting content tabs. But with a special query parameter we could target a chrome debugger server. I'm currently working on this for browser.html.

By default, as being a chrome:// URL it is going to load in parent process I think.
But if toolbox.xul is able to work in the content process, we could follow up with some magic to make it work in the content process...

Comment 2

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Alexandre Poirot [:ochameau] from comment #1)
> By default, as being a chrome:// URL it is going to load in parent process I
> think.
> But if toolbox.xul is able to work in the content process, we could follow
> up with some magic to make it work in the content process...

We can alter our DevTools chrome manifest to allow[1] loading chrome://devtools in the child by adding "remoteenabled=yes".  Of course it may still explode in other ways... but that at least lets the child attempt it.

[1]: https://dxr.mozilla.org/mozilla-central/source/chrome/nsIChromeRegistry.idl#79

Comment 3

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Filter on TEAPOT-SPLINES.

Comment 4

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

I experimented with loading the toolbox UI in the child, but it fails pretty fundamentally at first because XUL is not allowed in the child.

I was able to stub out a few things and get most of the UI to appear, but things like XUL <menupopup>s don't show up.  I may continue tinkering with the idea.

Overall though, we may have to wait until the conversion to HTML documents to run in the child.

Alternatively, we could run in the parent process if we leave out the debugger tab (since pausing would halt the same process containing the UI).

Comment 5

[Brian Grinstead [:bgrins]]

Sounds like a promising start, thanks for doing that.  Can you attach your current work so we have a reference for this once the toolbox frame is in HTML?

Comment 6

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Since this is just an experiment, here's an ongoing branch for it:

https://github.com/jryans/gecko-dev/commits/bt-child-proc

It's unclear how much of this would really be used, so I don't think it's ready to break out into bug attachments quite yet.

Comment 7

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Another idea :bgrins and I discussed on IRC was we could offer a toolbox UI running in the parent process but without the debugger tool (or alternatively only the inspector tool), which would address the desire for a quick loading browser inspector to use in place of the DOM Inspector add-on.

Comment 8

[Alexandre Poirot [:ochameau]]

Note that thanks to bug 1233463, it is already possible to load a Browser Toolbox in a tab via:
  about:devtools-toolbox?type=process
And a Browser Content Toolbox via:
  about:devtools-toolbox?type=process&id=1

You "just" have to be careful when using the Debugger panel.

Comment 9

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Alexandre Poirot [:ochameau] from comment #8)
> Note that thanks to bug 1233463, it is already possible to load a Browser
> Toolbox in a tab via:
>   about:devtools-toolbox?type=process
> And a Browser Content Toolbox via:
>   about:devtools-toolbox?type=process&id=1
> 
> You "just" have to be careful when using the Debugger panel.

Yes, that's exactly what I used to prototype the in process version in my branch above, thanks for working on that Alex. :)

So, we could potentially run that from a window and remove the debugger, if we wanted to.

Comment 10

[Brian Grinstead [:bgrins]]

(In reply to Alexandre Poirot [:ochameau] from comment #8)
> Note that thanks to bug 1233463, it is already possible to load a Browser
> Toolbox in a tab via:
>   about:devtools-toolbox?type=process
> And a Browser Content Toolbox via:
>   about:devtools-toolbox?type=process&id=1
> 
> You "just" have to be careful when using the Debugger panel.

This is great, didn't realize this was already supported!  So, we'd want to wire this up into a new window so it doesn't inspect itself and also disable debugger and maybe other tools.  And we want to be careful about adding 'yet another toolbox', but if we released that as an addon or as a super experimental thing to get feedback from fx team that could be a great way to gather feedback for DOMi replacement as in Comment 7.

Long term we still need to remove XUL so we can try to load it up in child process before we could 'replace' Browser Toolbox (at least for everyone except for DevTools devs who need to debug DT actors).

Comment 11

[Brian Grinstead [:bgrins]]

Another thought about Comments 7 - 10: we could possibly introduce lightweight version of the toolbox in place of the Browser Console (like console, inspector, and network).  That would avoid needing to add 'yet another toolbox'.  Downsides: we'd be introducing extra bugs / surface area into the BC.  Also I'm not sure if we have other non Firefox consumers of the Browser Console that would break from this change.

Comment 12

[Alexandre Poirot [:ochameau]]

I don't really follow why you are looking for alternatives? And especially looking for different implementation.
Couldn't we just fix/improve the existing browser toolbox?
I have the feeling that the current behavior is frustrating for 3 reasons:

1/ the connection prompt! We should really get rid of it, it makes no sense. Ok we should find a way to stay safe, but really that's so annoying and as soon as your discover the pref, you just end up toggling it and end up unsafe.

2/ default panel is the debugger, which is very slow to load (I think there is a bug opened for that). I'm convinced we can make it faster. It would be part of our dogfooding work as it would certainly make the web debugger also faster! What about making the console be the default, or the inspector?

3/ overall slowness. It takes time to spawn another gecko process and our tools are slower when debugging the parent process. but again, we could certainly tune gecko prefs to be faster (like not checking updates, sometimes we get some update dialogs...) and may be tweak the toolbox behavior to be more lazy and load faster.

At the end, no matter what we do, to have a decent debugger we would have to run in another process.
So unless we pull the debugger out, and end up with toolboxes panoply, we have to keep this browser toolbox running like this.

But I must admit I like the idea of getting rid of the special Browser Console and instead promote the Browser toolbox. We wouldn't think twice about this if the Browser toolbox was efficient! It would reduce the toolboxes panoply to just the web toolboxes, the about:debugging ones and the browser one.
CTRL+MAJ+J would open the browser toolbox to the console,
CTRL+ALT+MAJ+I would open the browser toolbox to the inspector (and may be auto inspect node?)

Comment 13

[Brian Grinstead [:bgrins]]

(In reply to Alexandre Poirot [:ochameau] from comment #12)
> I don't really follow why you are looking for alternatives? And especially
> looking for different implementation.

I think that's what this entire bug is about, so reasoning starting in Comment 0.  Some of the main points are perf (especially startup time), testability (which should help with stability aka breakpoints having problems lately in Browser Toolbox), and makes it easy to drop connection prompt.  Also easy to open up on multiple running Firefox instances without needing to listen and connect with non-default different port numbers (right now all BTs open targeting the first one's process by default).

> Couldn't we just fix/improve the existing browser toolbox?
> I have the feeling that the current behavior is frustrating for 3 reasons:
> 
> 1/ the connection prompt! We should really get rid of it, it makes no sense.
> Ok we should find a way to stay safe, but really that's so annoying and as
> soon as your discover the pref, you just end up toggling it and end up
> unsafe.

Yes, Bug 1013379 is opened for this and it would be great.

> 2/ default panel is the debugger, which is very slow to load (I think there
> is a bug opened for that). I'm convinced we can make it faster. It would be
> part of our dogfooding work as it would certainly make the web debugger also
> faster! What about making the console be the default, or the inspector?

I believe it should be defaulting to whatever was last opened by the Browser Toolbox profile.

> 3/ overall slowness. It takes time to spawn another gecko process and our
> tools are slower when debugging the parent process. but again, we could
> certainly tune gecko prefs to be faster (like not checking updates,
> sometimes we get some update dialogs...) and may be tweak the toolbox
> behavior to be more lazy and load faster.

I'd love if we could significantly speed up start up the time of a new firefox process but I suspect that's been optimized quite a lot already and there probably aren't many easy wins there.

> At the end, no matter what we do, to have a decent debugger we would have to
> run in another process.
> So unless we pull the debugger out, and end up with toolboxes panoply, we
> have to keep this browser toolbox running like this.

The idea is we might be able to even run the Debugger tool for everything *except* devtools actors by running it in the child process (which is dependent on XUL removal in the frontend)

Comment 14

[Alexandre Poirot [:ochameau]]

(In reply to Brian Grinstead [:bgrins] from comment #13)
> (In reply to Alexandre Poirot [:ochameau] from comment #12)
> > I don't really follow why you are looking for alternatives? And especially
> > looking for different implementation.
> 
> I think that's what this entire bug is about, so reasoning starting in
> Comment 0.

The connection prompt can be fixed without changing the process model.

> Some of the main points are perf (especially startup time),
> testability

I proved we can have tests against the browser toolbox. We now have one. And I'm currently trying to get one for the addon toolbox which is similar.

> (which should help with stability aka breakpoints having
> problems lately in Browser Toolbox),

I think we are just bad at dogfooding. We shouldn't let a bug live just because it is only in the browser toolbox. It very likely hide a bug that also happens in remote connections or web sites.

> and makes it easy to drop connection
> prompt.

Again, there is ways to mitigate that.

> Also easy to open up on multiple running Firefox instances without
> needing to listen and connect with non-default different port numbers (right
> now all BTs open targeting the first one's process by default).

Port numbers are just an implementation detail. I don't see why the browser toolbox even use static one, it should be random. Note that on linux and may be also on Mac we could be using domain socket to mitigate the security issue and workaround TCP port being already busy.

> > 2/ default panel is the debugger, which is very slow to load (I think there
> > is a bug opened for that). I'm convinced we can make it faster. It would be
> > part of our dogfooding work as it would certainly make the web debugger also
> > faster! What about making the console be the default, or the inspector?
> 
> I believe it should be defaulting to whatever was last opened by the Browser
> Toolbox profile.

But it would be great to have shortcut like domi, where the BT opens on the inspector and inspect hovering nodes!

> > 3/ overall slowness. It takes time to spawn another gecko process and our
> > tools are slower when debugging the parent process. but again, we could
> > certainly tune gecko prefs to be faster (like not checking updates,
> > sometimes we get some update dialogs...) and may be tweak the toolbox
> > behavior to be more lazy and load faster.
> 
> I'd love if we could significantly speed up start up the time of a new
> firefox process but I suspect that's been optimized quite a lot already and
> there probably aren't many easy wins there.

browser.xul is bloated! we are not loading it and it should be significantly faster to load than firefox!! I don't think anyone tried to tweak our profile/prefs to make it faster by disable all unecessary stuff like addons and similar.

Please take a close look at performances.
You will see that a empty window appears quickly. The process isn't that long to start. Then you see it empty for quite a while before the panel buttons appear and then you have to wait for another while for the panel to be ready.
Now, open about:devtools-toolbox?type=process in a tab. You will see that it takes about the same time to be ready. 

My take on this is that the tools are slow to load. We have much more time spent waiting for the tools to be ready rather than waiting for gecko to start.

I've spent monthes to lazify the server part, I'm convinced there is similar amount of work to be spent on the client side, especially now that we enable console, debugger and netmon by default...

> > At the end, no matter what we do, to have a decent debugger we would have to
> > run in another process.
> > So unless we pull the debugger out, and end up with toolboxes panoply, we
> > have to keep this browser toolbox running like this.
> 
> The idea is we might be able to even run the Debugger tool for everything
> *except* devtools actors by running it in the child process (which is
> dependent on XUL removal in the frontend)

Yes, that would be great. But we are far, very far from being there yet.
We should only consider that once we managed to switch most of our stuff on HTML.
It is too early to depend on that.