1152839 - Identify idle nodes in performance tool

Status: RESOLVED WONTFIX

(DevTools :: Performance Tools (Profiler/Timeline), defect, P3)

Description

[Jordan Santell [:jsantell] [@jsantell]]

Startup::XRE_InitChildProcess (e10s) and Startup::XRE_Main (non e10s) frames should be translated to "(idle)", as these take up the majority of the time when no explicit events occur.

We should probably get confirmation that this is the case for these frames as well.

Comment 1

[Jordan Santell [:jsantell] [@jsantell]]

Pinging for info to see if the above functions can be considered "idle" when visualizing profiler data, and if you're aware of other functions that can be considered idle. Thanks!

Comment 2

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

What does "idle" mean here?  I assume you mean that we're blocked in something like pthread_cond_wait or WaitMessageW?

Comment 3

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #2)
> What does "idle" mean here?  I assume you mean that we're blocked in
> something like pthread_cond_wait or WaitMessageW?

I think we are looking for what C++ frames we will see from the SPS profiler when the event queue is empty and just waiting for user input or some runnable or whatever.

Startup::XRE_Main is what I see most time being attributed to when I open a blank tab and profile nothing for a few seconds.

Comment 4

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

We should probably put in a specific frame for "waiting for events" then, because Startup::XRE_Main will be on the stack for everything.

Comment 5

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

We would only consider time idle if this was the youngest frame.

Comment 6

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Sure, but do we have a guarantee that something else will be on the stack?

Comment 7

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #6)
> Sure, but do we have a guarantee that something else will be on the stack?

I'm not sure what you're getting at?

If nothing is on the stack, then the process can't be running, can it?

If the youngest frame is Startup::XRE_main, then we will consider it idle (assuming that is the waiting-for-stuff frame).

If the youngest frame is not Startup::XRE_main, then we attribute the time to that youngest frame's function.

Am I missing something?

Comment 8

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

My question is whether we know that there will be something else on the SPS stack when we are in fact not idle.  Do we push something onto the stack for all events, or only "interesting" ones?

Comment 9

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #8)
> My question is whether we know that there will be something else on the SPS
> stack when we are in fact not idle.  Do we push something onto the stack for
> all events, or only "interesting" ones?

We have a catchall grouping for C++ frames that aren't known to be part of layout or graphics or whatever called "Gecko". Right now, idle time gets grouped into "Gecko", but the goal of this bug is to split it out.

The only tricky scenario I can think of is if there are many multiple functions that when they are the younggest frame on the stack mean "we are idle". Any C++ frames we haven't explicitly labeled idle and don't belong to another category will fall under "Gecko" by default.

Comment 10

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Do you actually see the C stack, or just the psuedostack?  If you see the real C stack then my concerns are probably moot.

Comment 11

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

Attachment: screen shot of idle time being grouped as "Gecko"

Comment 12

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #10)
> Do you actually see the C stack, or just the psuedostack?  If you see the
> real C stack then my concerns are probably moot.

I'm not familiar enough with the profiler. ni djvj

Comment 13

[Kannan Vijayan [:djvj]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #10)
> Do you actually see the C stack, or just the psuedostack?  If you see the
> real C stack then my concerns are probably moot.

No, you don't see the C stack.  Just the pseudostack + js stack.

Comment 14

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Kannan Vijayan [:djvj] from comment #13)
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #10)
> > Do you actually see the C stack, or just the psuedostack?  If you see the
> > real C stack then my concerns are probably moot.
> 
> No, you don't see the C stack.  Just the pseudostack + js stack.

Talking with Jordan on IRC, I came to the opposite conclusion: IIUC we are grouping by C stack but displaying with the psuedo stack label b/c we don't do symbolication yet (so you could get "duplicate" frames with the same psuedo stack label but for different native frames).

Comment 15

[Shu-yu Guo [:shu]]

(In reply to Kannan Vijayan [:djvj] from comment #13)
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #10)
> > Do you actually see the C stack, or just the psuedostack?  If you see the
> > real C stack then my concerns are probably moot.
> 
> No, you don't see the C stack.  Just the pseudostack + js stack.

What is the context of "see" here? Visually, in the Performance Tool, or whether or not C stacks are visible in the underlying JSON data of the profile?

With "stackwalk", we would have C stacks, no?

Comment 16

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

My point, in case it hasn't been clear, is that if you are looking at the psuedostack it may be possible that there are things that simply have not been marked for SPS that will not add any pseudostack entries when they are running but are still not idle.

Comment 17

[Shu-yu Guo [:shu]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #16)
> My point, in case it hasn't been clear, is that if you are looking at the
> psuedostack it may be possible that there are things that simply have not
> been marked for SPS that will not add any pseudostack entries when they are
> running but are still not idle.

Right, that's a legit concern. I am under the impression we actually have C stack walking and report such stacks if the right features in nsProfiler are requested when starting to profile. Without these I don't see how it's possible to approximate % idle.

Comment 18

[Kannan Vijayan [:djvj]]

(In reply to Shu-yu Guo [:shu] from comment #15)
> What is the context of "see" here? Visually, in the Performance Tool, or
> whether or not C stacks are visible in the underlying JSON data of the
> profile?
> 
> With "stackwalk", we would have C stacks, no?

Yeah, they'd be included in the JSON when the 'stackwalk' option is passed to the profiler start API.  However the builtin tool won't visualize it.

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #16)
> My point, in case it hasn't been clear, is that if you are looking at the
> psuedostack it may be possible that there are things that simply have not
> been marked for SPS that will not add any pseudostack entries when they are
> running but are still not idle.

If we push an "Idle" SPS entry when waiting for events on the event loop, that should be reliable, no?

Comment 19

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Kannan Vijayan [:djvj] from comment #18)
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #16)
> > My point, in case it hasn't been clear, is that if you are looking at the
> > psuedostack it may be possible that there are things that simply have not
> > been marked for SPS that will not add any pseudostack entries when they are
> > running but are still not idle.
> 
> If we push an "Idle" SPS entry when waiting for events on the event loop,
> that should be reliable, no?

Yes.  See comment 4.

Comment 20

[Jordan Santell [:jsantell] [@jsantell]]

So what's the verdict for this? SPS will generate idle entries? Who can help with that?

We'd like to have this for the 40.1 release, or we can do what Nick mentioned with relabeling the XRE_Main leaf frames as "idle" on the front end for a quicker (?) result -- less accurate than what we have now, but arguably more correct.

Comment 21

[Jordan Santell [:jsantell] [@jsantell]]

Some discussion in faking this on the front in bug 1167976#c5; resolving to continue to want to handle this on the profiler end

Comment 23

[Greg Tatum [:gregtatum]]

I believe this discussion is no longer relevant, and idle issues have been discussed on the perf.html project. Here is one particular one to track: https://github.com/devtools-html/perf.html/issues/598