Closed Bug 809030 Opened 12 years ago Closed 12 years ago

B2G crash reports missing symbols

Categories

(Firefox OS Graveyard :: General, defect)

Product:
Firefox OS Graveyard
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
B2G C1 (to 19nov)

People

(Reporter: akeybl, Assigned: ted)

References

Details

Sheila emailed yesterday that "I was just told we don't have the symbols coming through yet. When we we expect to see these." Jgriffin responded "Can you connect whoever you told you this to me, so we can iron out what's going wrong?". Given that, assigning to Ted.

Additionally, I'm marking this bug with the C1 milestone since it follows the criteria of "Crash feedback must be coming in and actionable from dogfooders".

If this work is not finished by Nov19, this bug will need an exception and will be called out at the upcoming Exec Review.
From all we know, this bug is probably WFM in that symbols are said to be uploaded fine. All we need to do is QA that to verify or dismiss the claims that everything is alright.

Actually, submitting crash reports might be broken due to the recent changes with the crash reporting UI, we also need to QA that (and make sure they are only sent on wifi, not on mobile data, but are being queued up when not on wifi and sent as soon as we are on wifi - I'm pretty sure the latter isn't implemented, but we need that).
It looks like we have symbols on the symbol server:
tmielczarek@symbols1.dmz.phx1 symbols_b2g]$ pwd
/mnt/netapp/breakpad/symbols_b2g
[tmielczarek@symbols1.dmz.phx1 symbols_b2g]$ ls
b2g                                                libplds4.so
b2g-18.0a2-Android-20121031073004-arm-symbols.txt  libsmime3.so
...

Can someone provide me more info about how "we don't have symbols coming through yet" was determined?
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> Can someone provide me more info about how "we don't have symbols coming
> through yet" was determined?

That might come from my statement in the meeting that we haven't seen symbolized crashes from devices on crash-stats so far. But I also had problems on finding reports from recent unagi builds at all.
Okay. I've verified that symbols are being uploaded. I can't do anything more here without more information. If you start getting new crash reports and the symbols are not present then please reopen this.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → INCOMPLETE
Actually, I'll reopen so more investigation can be done. I pulled all current B2G reports from devices into a list (will now be done daily), see https://crash-analysis.mozilla.com/rkaiser/2012-11-05/2012-11-05.b2g.crashes.html - and I don't see any of the unagi crashes having symbols. Could you see if we have symbols for the builds those crashed with?
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
So, these crashes come from a build that has had symbols uploaded:

https://crash-stats.mozilla.com/report/list?signature=%400x4105b0c0
https://crash-stats.mozilla.com/report/list?signature=libmedia.so%400x3b792

Someone with MPT access can see the build logs which include the upload here:

http://builder1.ateam.phx1.mozilla.com/job/build-unagi/88/console

It looks to me like the upload succeeded.
Depends on: 809116
So at least part of the problem is bug 809116. When that's fixed we should have symbols for the Gecko libraries in the stack. I verified that the symbol files for the Gecko libraries in those crash reports are present on the symbol server.

We still won't have symbols for system libraries until I finish fixing bug 774284. I got a bit derailed by our team work week, I'll get back to that.
Summary: Crash symbols are not being uploaded (and thus aren't actionable) → B2G crash reports missing symbols
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #7)
> I verified that the
> symbol files for the Gecko libraries in those crash reports are present on
> the symbol server.

But aren't things like libmozglue.so Gecko libraries? I don't see Socorro resolving those symbols then. :(
Does the changes only affect nightly channel?  I think the builds are based off of aurora channel...
Any gecko-based changes will have to land in aurora before they affect the nightly/stable unagi builds.
There are no Gecko changes being discussed here. bug 809116 is a Socorro configuration change.
(In reply to Naoki Hirata :nhirata from comment #9)
> Does the changes only affect nightly channel?  I think the builds are based
> off of aurora channel...

What jgriffin fixed affects whatever builds he applies the process to, channels don't really play a role there, his build process does. The Socorro configuration change automatically applies to any symbols we get for B2G, independent of any channel.

And, BTW, we currently send the "wrong" channel settings in builds (AFAIK "nightly" even for builds off aurora), but that will be dealt with in yet another bug.
This is fixed (although it's hard to notice given our low crash volume). For example:
https://crash-stats.mozilla.com/report/index/338c5f80-41d5-4d33-b57a-b8f462121111
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.