Closed
Bug 1069702
Opened 10 years ago
Closed 10 years ago
Linux Mulet nightlies failing during the uploadsymbols step of the build process
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: RyanVM, Assigned: jgriffin)
References
Details
Attachments
(1 file)
734 bytes,
patch
|
nthomas
:
review+
jgriffin
:
checked-in+
|
Details | Diff | Splinter Review |
These were enabled today and are failing on trunk and Aurora. https://tbpl.mozilla.org/php/getParsedLog.php?id=48423150&tree=Mozilla-Central Linux x86-64 Mulet mozilla-central nightly on 2014-09-18 17:09:06 PDT for push c8e325eee9e1 slave: b-linux64-ix-0008 INFO: mock_mozilla.py version 1.0.3 starting... State Changed: init plugins INFO: selinux disabled State Changed: start State Changed: lock buildroot State Changed: shell retry: Calling <function run_with_timeout at 0x1d5e668> with args: (['make', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #1 Executing: ['make', 'uploadsymbols'] /bin/sh /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-35.0a1-Linux-20140918160202-x86_64-symbols.txt 'dist/firefox-35.0a1.en-US.linux-x86_64.crashreporter-symbols-full.zip' /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh: line 26: SYMBOL_SERVER_HOST: parameter null or not set make: *** [uploadsymbols] Error 1 retry: Failed, sleeping 1 seconds before retrying retry: Calling <function run_with_timeout at 0x1d5e668> with args: (['make', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #2 Executing: ['make', 'uploadsymbols'] /bin/sh /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-35.0a1-Linux-20140918160202-x86_64-symbols.txt 'dist/firefox-35.0a1.en-US.linux-x86_64.crashreporter-symbols-full.zip' /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh: line 26: SYMBOL_SERVER_HOST: parameter null or not set make: *** [uploadsymbols] Error 1 retry: Failed, sleeping 2 seconds before retrying retry: Calling <function run_with_timeout at 0x1d5e668> with args: (['make', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #3 Executing: ['make', 'uploadsymbols'] /bin/sh /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-35.0a1-Linux-20140918160202-x86_64-symbols.txt 'dist/firefox-35.0a1.en-US.linux-x86_64.crashreporter-symbols-full.zip' /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh: line 26: SYMBOL_SERVER_HOST: parameter null or not set make: *** [uploadsymbols] Error 1 retry: Failed, sleeping 4 seconds before retrying retry: Calling <function run_with_timeout at 0x1d5e668> with args: (['make', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #4 Executing: ['make', 'uploadsymbols'] /bin/sh /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-35.0a1-Linux-20140918160202-x86_64-symbols.txt 'dist/firefox-35.0a1.en-US.linux-x86_64.crashreporter-symbols-full.zip' /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh: line 26: SYMBOL_SERVER_HOST: parameter null or not set make: *** [uploadsymbols] Error 1 retry: Failed, sleeping 8 seconds before retrying retry: Calling <function run_with_timeout at 0x1d5e668> with args: (['make', 'uploadsymbols'], 2460, None, None, False, True), kwargs: {}, attempt #5 Executing: ['make', 'uploadsymbols'] /bin/sh /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh firefox-35.0a1-Linux-20140918160202-x86_64-symbols.txt 'dist/firefox-35.0a1.en-US.linux-x86_64.crashreporter-symbols-full.zip' /builds/slave/m-cen-l64-mulet-ntly-000000000/build/toolkit/crashreporter/tools/upload_symbols.sh: line 26: SYMBOL_SERVER_HOST: parameter null or not set make: *** [uploadsymbols] Error 1 retry: Giving up on <function run_with_timeout at 0x1d5e668> Unable to successfully run ['make', 'uploadsymbols'] after 5 attempts
Comment 1•10 years ago
|
||
The failure above is uploading to Socorro, which seems of little utility to me but I don't really how widely these builds are, or will be, used. Perhaps we only need SYMBOL_SERVER_HOST files for tests ? The relevant code is at http://hg.mozilla.org/build/buildbotcustom/file/default/process/factory.py#l1077: 1077 if self.uploadSymbols or (not self.disableSymbols and self.packageTests): 1078 self.addBuildSymbolsStep() 1079 if self.uploadSymbols: 1080 self.addUploadSymbolsStep() The linux-mulet platform has upload_symbols set to True in b2g_config.py, and self.uploadSymbols ends up True. We already have settings to make the 2nd half of the or true, so the way forward is * if we don't need Socorro upload, change http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla/b2g_config.py#l406 to False * if we do want Socorro, then see http://hg.mozilla.org/mozilla-central/file/default/toolkit/crashreporter/tools/upload_symbols.sh for what need to be set in the env (SYMBOL_SERVER_HOST), and figure out how to modify what we have in config.py to do that. We should use a different SYMBOL_SERVER_PATH at least.
Component: Buildduty → General Automation
QA Contact: bugspam.Callek → catlee
Summary: Mulet nightlies failing during the uploadsymbols step of the build process → Linux Mulet nightlies failing during the uploadsymbols step of the build process
Comment 2•10 years ago
|
||
(In reply to Nick Thomas [:nthomas] from comment #1) > Perhaps we only need SYMBOL_SERVER_HOST files for tests ? Oopsie, meant files like firefox-35.0a1.en-US.linux-x86_64.crashreporter-symbols.zip.
Assignee | ||
Comment 3•10 years ago
|
||
Currently, I'm thinking we don't need soccoro, so I'll do the simplest thing and turn the upload off. We can revisit later if we decide we want this.
Assignee | ||
Comment 4•10 years ago
|
||
Attachment #8492254 -
Flags: review?(nthomas)
Assignee | ||
Updated•10 years ago
|
Assignee: nobody → jgriffin
Comment 5•10 years ago
|
||
Comment on attachment 8492254 [details] [diff] [review] Turn off symbol uploads, Works as expected in a master dump.
Attachment #8492254 -
Flags: review?(nthomas) → review+
Assignee | ||
Comment 6•10 years ago
|
||
Comment on attachment 8492254 [details] [diff] [review] Turn off symbol uploads, Thanks for verifying. https://hg.mozilla.org/build/buildbot-configs/rev/00a5bfaa2bc5
Attachment #8492254 -
Flags: checked-in+
Comment 7•10 years ago
|
||
I'm seeing mulet symbolupload errors on try - i assume we should skip it there as well? E.g. https://tbpl.mozilla.org/?tree=Try&rev=a6cf446146c0
Assignee | ||
Comment 8•10 years ago
|
||
(In reply to Georg Fritzsche [:gfritzsche] from comment #7) > I'm seeing mulet symbolupload errors on try - i assume we should skip it > there as well? > E.g. https://tbpl.mozilla.org/?tree=Try&rev=a6cf446146c0 Yes, this will be fixed as soon as the patch for this bug goes into production.
Comment 9•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #3) > Currently, I'm thinking we don't need soccoro, so I'll do the simplest thing > and turn the upload off. We can revisit later if we decide we want this. We'd only need uploadsymbols run if we either: a) Intend to have users running these nightlies and want to accept crash reports or b) Want to allow users to debug (with a native debugger) the nightlies we're shipping using the symbol server
Comment 10•10 years ago
|
||
Something here landed in production today: https://wiki.mozilla.org/ReleaseEngineering/Maintenance#Reconfigs_.2F_Deployments
Assignee | ||
Updated•10 years ago
|
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•