Open Bug 1341811 Opened 7 years ago Updated 2 years ago

Consider hooking pthread_create (or another technique) so that we can profile threads launched by libraries


(Core :: Gecko Profiler, enhancement, P3)




Tracking Status
firefox54 --- affected


(Reporter: mstange, Unassigned)


(Depends on 1 open bug, Blocks 2 open bugs)


Threads launched by media libraries or WebRTC don't go through NS_NewNamedThread, so we don't automatically register them to the profiler.

We should investigate whether we can hook pthread_create (and the Windows equivalent) in order to automatically register all threads with the profiler.
Glandium had some ideas about this in Austin.
Flags: needinfo?(mh+mozilla)
Oh, my, this has been ni? for way too long. Sorry about that.

Windows is, interestingly, the easiest to handle. You just need a DllMain in one of our dlls (xul.dll), and have it handle the DLL_THREAD_ATTACH event. We have that in toolkit/library/nsDllMain.cpp.

On linux, we can export a pthread_create wrapper from our executable, but that won't catch all thread creations. Anything using the system call directly (and there may be) won't be caught. I guess that's still better than nothing?

On mac, we can wrap pthread_create, but that won't catch any threads created by system libraries, and there are. OTOH, OSX has APIs to enumerate all threads:
Flags: needinfo?(mh+mozilla)
Also note threads can be enumerated on linux by reading the directory entries in /proc/self/task. Arguably, on OSes where we can enumerate threads, we don't need to have them registered to the profiler.

For what it's worth:

  • unnamed threads are now visible in about:processes;
  • we can easily test that all threads are OS-level named in the unit tests for about:processes.

Unfortunately, as of today, there are still a few unnamed threads. I'm tracking them as blockers of bug 1630898.

Blocks: 1630898

Thank you David.

Obviously about:processes must have a way to get the list of threads (ids and names), I hope the profiler could use the same technique or even the same code, in which case it should be possible to record them.

I think this bug here should depend on bug 1630898 instead of blocking it (i.e., the profiler depends on threads being named before it can profile them) -- But please let me know if you disagree.

No longer blocks: 1630898
Type: defect → enhancement
Depends on: 1630898
Summary: Consider hooking pthread_create so that we can profile threads launched by libraries → Consider hooking pthread_create (or another technique) so that we can profile threads launched by libraries

Oh, while it's great if we can get a list of threads, I'm guessing it may not be cheap to do that, so the profiler may not be able to do it at every sample point!
If my assumption is sadly correct we will still want to try and catch thread creations (by hooking pthread_create or other means).

But in the meantime, it could still be valuable to have an option to regularly (1 Hz?) look at the list of threads, and profile new ones at they appear.

And on the flip side, we'll want to catch thread terminations; but that should (hopefully) be trivial as they'll just not be there anymore when we try to sample them. (Another assumption: Thread ids won't be reused in-between two samplings.)

Fission Milestone: --- → M7
Fission Milestone: M7 → ---

Yoric (or whomever): where are threads in about:processes? Also, we're talking raw threads/pthreads, not nsThreads which we can already enumerate.

Flags: needinfo?(D.O.Teller+bugspam)

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #7)

Yoric (or whomever): where are threads in about:processes?

They are shown if you set toolkit.aboutProcesses.showThreads to true in about:config.

Flags: needinfo?(D.O.Teller+bugspam)

They are hidden by default. You need pref toolkit.aboutProcesses.showThreads = true to show them.

See Also: → 1724420

I've just filed bug 1742522, which will discover unregistered threads the same way about:processes does.
It should help with profiling library-controlled threads, but not as soon as threads are started, so I will keep this bug here open in case we can actually hook pthread_create (or equivalent/better, and equivalent on other platforms).

Depends on: 1742522

Note that the thread discovery in about:processes is pretty slow.

(In reply to David Teller [:Yoric] - still alive but not very active from comment #11)

Note that the thread discovery in about:processes is pretty slow.

True, it's not be perfect but hopefully it will give some useful data on all platforms, until someone can get this bug here done!

Severity: normal → N/A
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.