When building Chromium-as-release, run source_index.py after the build is done so that the PDBs contain better file paths
Categories
(Testing :: Performance, enhancement)
Tracking
(Not tracked)
People
(Reporter: mstange, Unassigned)
References
Details
(Whiteboard: [fxp])
The chromium repo contains a script at tools/symsrc/source_index.py which, when run, improves the symbol information in the PDB files. Could we run this tool at the end of the chromium-as-release build?
This would let us show source code in the profiler, in profiles collected from these builds.
It's used as follows:
For each file matching out/Default/*.pdb (but most importantly for chrome.dll.pdb):
python tools/symsrc/source_index.py --build-dir out/Default --toolchain-dir <toolchain-dir-goes-here> --exclusion-dirs "" thefile.pdb
It expects the tools srctool.exe and pdbstr.exe to be available at toolchain_dir/Windows Kits/10/Debuggers/x64/srcsrv .
|
Reporter | |
Comment 1•17 days ago
|
||
The script also runs git.bat to find the repository and revision for a file.
|
||
Updated•9 days ago
|
|
||
Updated•9 days ago
|
|
||
Comment 2•9 days ago
|
||
should be quick to add to a Try run, let me give it a shot
|
|
||
Comment 3•9 days ago
•
|
||
:mstange there doesn't seem to be a srcsrv directory in toolchain_dir/Windows Kits/10/Debuggers/x64/srcsrv directory e.g. https://firefox-ci-tc.services.mozilla.com/tasks/LPgTyjTSSq-aCDS62cs30Q/runs/0/logs/live/public/logs/live.log#L1009-1150 (the task is still running as of this comment so looking at the live log might keep jumping around)
Is it possible something is missing in the update SDK toolchain in Bug 1925145?
|
|
Reporter | |
Comment 4•9 days ago
|
||
I can point you at bug 1618781 which changed how Firefox builds get pdbstr.exe. I don't really understand how all this works.
|
|
Reporter | |
Comment 5•9 days ago
|
||
If the outcome of this investigation is "the script makes too many assumptions that would be hard to satisfy in our environment" that's ok. Maybe we can write our own, simplified, version of it.
|
|
||
Comment 6•9 days ago
|
||
(In reply to Markus Stange [:mstange] from comment #4)
I can point you at bug 1618781 which changed how Firefox builds get pdbstr.exe. I don't really understand how all this works.
Okay so I copied over the debugger stuff that was done in bug 1884231 and that srcsrv directory exists now and inside I see this:
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 838 Oct 24 19:09 cv2http.cmd
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 1036 Oct 24 19:09 cv2http.pl
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 19545 Oct 24 19:09 cvs.pm
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 188 Oct 24 19:09 cvsindex.cmd
[task 2024-10-24T19:27:39.881Z] -rwxr-xr-x 1 task_172979723416851 None 175056 Oct 24 19:09 dbgcore.dll
[task 2024-10-24T19:27:39.881Z] -rwxr-xr-x 1 task_172979723416851 None 1869264 Oct 24 19:09 dbghelp.dll
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 14061 Oct 24 19:09 p4.pm
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 187 Oct 24 19:09 p4index.cmd
[task 2024-10-24T19:27:39.881Z] -rwxr-xr-x 1 task_172979723416851 None 996304 Oct 24 19:09 pdbstr.exe
[task 2024-10-24T19:27:39.881Z] -rwxr-xr-x 1 task_172979723416851 None 143312 Oct 24 19:09 srcsrv.dll
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 4515 Oct 24 19:09 srcsrv.ini
[task 2024-10-24T19:27:39.881Z] -rwxr-xr-x 1 task_172979723416851 None 43984 Oct 24 19:09 srctool.exe
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 32964 Oct 24 19:09 ssindex.cmd
[task 2024-10-24T19:27:39.881Z] -rw-r--r-- 1 task_172979723416851 None 12509 Oct 24 19:09 svn.pm
[task 2024-10-24T19:27:39.886Z] -rw-r--r-- 1 task_172979723416851 None 188 Oct 24 19:09 svnindex.cmd
[task 2024-10-24T19:27:39.886Z] -rw-r--r-- 1 task_172979723416851 None 19415 Oct 24 19:09 tfs.pm
[task 2024-10-24T19:27:39.886Z] -rw-r--r-- 1 task_172979723416851 None 188 Oct 24 19:09 tfsindex.cmd
[task 2024-10-24T19:27:39.886Z] -rw-r--r-- 1 task_172979723416851 None 13846 Oct 24 19:09 vss.pm
[task 2024-10-24T19:27:39.886Z] -rw-r--r-- 1 task_172979723416851 None 188 Oct 24 19:09 vssindex.cmd
[task 2024-10-24T19:27:39.886Z] -rw-r--r-- 1 task_172979723416851 None 343 Oct 24 19:09 walk.cmd
I see the srctool.exe and pdbstr.exe there
the task is still running so we'll see!
|
|
Reporter | |
Comment 7•9 days ago
|
||
(In reply to Kash Shampur [:kshampur] ⌚EST from comment #6)
(In reply to Markus Stange [:mstange] from comment #4)
I can point you at bug 1618781 which changed how Firefox builds get pdbstr.exe. I don't really understand how all this works.
Okay so I copied over the debugger stuff that was done in bug 1884231
oh oops, yes that was the bug that I actually meant to point to. Glad you found it!
|
|
Reporter | |
Comment 8•9 days ago
|
||
er what, those numbers are the same, I got confused. nevermind
|
|
||
Comment 9•7 days ago
|
||
hitting this error
[task 2024-10-25T17:14:06.613Z] + python3 tools/symsrc/source_index.py --build-dir out/Default --toolchain-dir D:/task_172986304147815/fetches/VS --exclusion-dirs '' out/Default/VkICD_mock_icd.dll.pdb
[task 2024-10-25T17:14:06.835Z] Traceback (most recent call last):
[task 2024-10-25T17:14:06.835Z] File "tools/symsrc/source_index.py", line 604, in <module>
[task 2024-10-25T17:14:06.835Z] sys.exit(main())
[task 2024-10-25T17:14:06.835Z] File "tools/symsrc/source_index.py", line 597, in main
[task 2024-10-25T17:14:06.835Z] UpdatePDB(pdb, options.verbose, options.build_dir, options.toolchain_dir,
[task 2024-10-25T17:14:06.835Z] File "tools/symsrc/source_index.py", line 509, in UpdatePDB
[task 2024-10-25T17:14:06.835Z] filelist = ExtractSourceFiles(pdb_filename, toolchain_dir)
[task 2024-10-25T17:14:06.835Z] File "tools/symsrc/source_index.py", line 174, in ExtractSourceFiles
[task 2024-10-25T17:14:06.835Z] raise Exception("srctool failed: " + src_files)
[task 2024-10-25T17:14:06.835Z] Exception: srctool failed:
srctool failing... src_files seemes to return as empty from here https://github.com/chromium/chromium/blob/0860b79d71c01a222bc4d61d6eb021b1f8ed759d/tools/symsrc/source_index.py#L170
FindSrcSrvFile('srctool.exe', toolchain_dir) call seems to be right otherwise this assertion would fail... Maybe there is permissions error in trying to use srctool.exe. But i don't think that should happen... can look closer next week.
Description
•