Closed Bug 1253070 Opened 4 years ago Closed 3 years ago
_CRASHREPORTER _UPLOAD _FULL _SYMBOLS option on by default
When downloadBuild.py downloads symbols, it produces the normal directory structures (\symbols\binary_name.pdb\[hash]\) but each folder contains a "binary_name.sym" file, rather than a "binary_name.pdb" file. I'm not sure what the former are, but I would expect the latter. So does WinDbg/cdb; I am getting no symbol information from these ".sym" files.
(In reply to SkyLined from comment #0) > When downloadBuild.py downloads symbols What you mean is probably the following file: https://ftp.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-win32/1456930532/firefox-47.0a1.en-US.win32.crashreporter-symbols.zip In this zip file, I see the firefox.pdb folder, and firefox.sym is located within the subfolders of this folder, which is probably what Skylined is referring to here. :glandium/Ted, do you have an idea what might be going on? (or feel free to pass on the baton to someone else more appropriate)
Component: Untriaged → Breakpad Integration
Product: Core → Toolkit
Guessing this is related to Breakpad.
This is expected behavior. We don't upload the native debug symbols alongside the build, because we don't use them in automation and don't want to waste the time uploading/downloading them there. We upload native debug symbols from nightly/release builds to the symbol server. (I don't know what downloadBuild.py is, FWIW.)
downloadBuild.py is merely a file in the fuzzing harness to download specific Firefox/js builds (as well as other files, including the above-mentioned firefox-47.0a1.en-US.win32.crashreporter-symbols.zip): https://github.com/MozillaSecurity/funfuzz/blob/29139149d01ac93d040d08d8bc6b11885ff0497a/util/downloadBuild.py downloadBuild.py isn't the issue here, though. Wrt. *.crashreporter-symbols.zip, will it make sense to start uploading native debug aurora/nightly/treeherder per-push symbols so fuzzers can use the builds for testing and get symbols? Is it too much overhead (space requirements) to ask? We could restrict to store only the past month's symbols, for example.
I think space requirements are probably manageable nowadays since all the builds are stored in S3. I still don't want to put the symbols into the .symbols.zip package, since as I stated in comment 3 we have to download that package in automation for debug builds and getting stacks out of crashes, and the native symbols add a lot of space (and thus overhead in downloading the file). What we could do is flip the MOZ_CRASHREPORTER_UPLOAD_FULL_SYMBOLS option on by default, which uploads a separate crashreporter-symbols-full.zip alongside the existing symbols.zip: https://dxr.mozilla.org/mozilla-central/search?tree=mozilla-central&q=MOZ_CRASHREPORTER_UPLOAD_FULL_SYMBOLS&redirect=true Your script could then fetch that file which would give you the debug symbols you need. (As an aside, you might want to consider switching to or relying on mozdownload, since it solves the same problem but is well-maintained: https://github.com/mozilla/mozdownload )
mozdownload in turn requires pip for installing, which seems to turn this into a chain of dependent tools and scripts I would need to run in sequence. This is much more fragile than a single, simple, stand-alone script I can run to download binary + symbols. If a separate file with full symbols could be made available in the same folder, I can easily update the code for downloadBuild.py to fetch that. This would solve my problem.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5) > What we could do is flip the > MOZ_CRASHREPORTER_UPLOAD_FULL_SYMBOLS option on by default, which uploads a > separate crashreporter-symbols-full.zip alongside the existing symbols.zip: > https://dxr.mozilla.org/mozilla-central/search?tree=mozilla- > central&q=MOZ_CRASHREPORTER_UPLOAD_FULL_SYMBOLS&redirect=true > > Your script could then fetch that file which would give you the debug > symbols you need. Updating summary since this is what we need.
Summary: Symbols only contain ".sym" files and no ".pdb" files. → Flip MOZ_CRASHREPORTER_UPLOAD_FULL_SYMBOLS option on by default
Can I assume flipping this option is not a very complex process and I can expect builds with debug symbols soon? I'd like to update the build I fuzz frequently to avoid reporting issues that have already been addressed. I'd also like to start reporting the issues I found so far. Both are not practically possible without symbols, as I cannot get a proper stack to automatically detect duplicates or manually analyze the crashes.
Is there anything I can do to help expedite this?
(In reply to SkyLined from comment #9) > Is there anything I can do to help expedite this? You could write and test the patch! It's probably 5 lines or so. Removing the ifdef here: https://dxr.mozilla.org/mozilla-central/rev/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/toolkit/mozapps/installer/upload-files.mk#491 and just moving that `$(call QUOTED_WILDCARD ...` line up into the UPLOAD_FILES list above, then removing the other reference to this variable for completeness: https://dxr.mozilla.org/mozilla-central/rev/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/old-configure.in#7515
Sorry, been too busy maintaining my own code to work on yours... will look into this next week.
Assignee: nobody → blassey.bugs
Attachment #8802299 - Flags: review?(ted)
forgot to qref
Comment on attachment 8802524 [details] [diff] [review] upload_full_symbols.patch Review of attachment 8802524 [details] [diff] [review]: ----------------------------------------------------------------- Thanks!
Attachment #8802524 - Flags: review?(ted) → review+
Pushed by email@example.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/5d3e8e87351c Flip MOZ_CRASHREPORTER_UPLOAD_FULL_SYMBOLS option on by default. r=ted
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/mozilla-inbound/rev/3e60f475551d followup, make it actually, you know, build
You need to log in before you can comment on or make changes to this bug.