1508215 - The graphics adapter and vendor names appear truncated and too many entries are shown

Status: RESOLVED FIXED

(Socorro :: Webapp, task, P2)

Description

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: Crash report

I noticed that when you open the "Graphics Adapter" table of the Summary view for a given signature both the name of the graphics vendor and the adapter name are truncated. More specifically it seems that if the name contains spaces or dashes it gets "cut up" around those and only one word per entry is shown. Additionally multiple entries are being shown for a single crash holding "permutations" of the words in the string. See this search query for example:

https://crash-stats.mozilla.com/signature/?product=FennecAndroid&adapter_device_id=rogue&signature=java.lang.NullPointerException%3A%20at%20android.view.ViewRootImpl.performDraw%28ViewRootImpl.java%29&date=%3E%3D2018-11-12T09%3A58%3A59.000Z&date=%3C2018-11-19T09%3A58%3A59.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_columns=adapter_device_id&_columns=adapter_vendor_id&_sort=-date&page=1#summary

It should show a single crash report. I've attached a screenshot of the entry. The adapter vendor is "Imagination Technologies" and the adapter name is "PowerVR Rogue GE8100".

In the second attachment the crash is shown as part of the summary page. The "Graphics Adapter" entry has 6 lines. The first three with "imagination" as the vendor name, and the following three with "technologies". The adapter names are either "ge8100", "powervr" or "rogue". There should be only one line with "Imagination Technologies" as the vendor name and "PowerVR Rogue GE8100" as the adapter name.

Comment 1

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: Summary page, graphics adapter table

Comment 2

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: Summary page, graphics adapter table

Comment 3

[Gabriele Svelto [:gsvelto] (PTO)]

Interestingly this seems to be happening specifically on entries for Fennec builds. For example I just looked at this query which contains a single Mac crash and the adapter vendor name and id are displayed correctly:

https://crash-stats.mozilla.com/signature/?build_id=20181117220111&product=Firefox&release_channel=nightly&platform=Mac%20OS%20X&signature=_QLSeamlessOpeningCallCompletionHandlerAndInvalidate&date=%3E%3D2018-11-17T02%3A00%3A00.000Z&date=%3C2018-11-19T10%3A22%3A40.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_sort=-date&page=1#summary

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Funky! I'll look into it.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

What's going on is that Fennec builds aren't sending ids (for example, "0x84")--they're sending full strings (for example, "Imagination Technologies"). The adapter_id and vendor_id fields are getting indexed such that when Socorro does a facet, the ids get parsed into terms so then the faceting returns "imagination" and "technology" as separate entries and then it looks wrong in the signature report.

I claim the Fennec crash reports should be sending hex ids in the adapter_id and vendor_id field and not names. But, here we are and it's probably the case that Socorro should get fixed in some way to deal with it. I'll look into changing how Socorro indexes those two fields. Maybe I can change it so it's not breaking names up into words without messing up faceting?

I think I can tinker with this next week.

Comment 6

[Gabriele Svelto [:gsvelto] (PTO)]

And I'll have a look at the Fennec code to see how it retrieves the vendor and adapter IDs and if raw versions are available.

Comment 7

[Gabriele Svelto [:gsvelto] (PTO)]

I've looked at Fennec's code and realized what's going on. On Android there's no such thing as PCI IDs for the various devices in the system because there's no PCI/PCIe subsystem in the first place! Mobile phone SoCs don't enumerate devices like PCs do so there's no way of getting IDs because there's none :( So Fennec's code retrieves the vendor and adapter *names* from the OpenGL driver instead. So I believe there's two ways to fix:

- On Soccorro we parse those fields and if they're not in the 0xNNNN format of an ID they assume that they're names and not IDs. But as you said that could break stuff on Socorro's side so a better fix may be...

- We add two new crash annotations "AdapterVendorName" and "AdapterDeviceName" and we use those on Fennec instead of "AdapterVendorID" and "AdapterDeviceID". This was it should be easier on Socorro's side to tell them apart. "AdapterVendorName" and "AdapterDeviceName" are already synthesized for non-Fennec builds so it would require changes on Socorro but at least this would be consistent with what we have now.

On this topic, I just noticed that we don't gather those fields on Linux either, time to file another bug.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

That makes a lot of sense!

I think it makes a lot of sense to do the first one (Socorro indexes the field values better). We can use this bug for that work.

On the second one (adding two new annotations), I don't have an opinion. I'm not sure if it makes Socorro maintenance harder or easier. I don't know if anyone is searching with those fields and whether having another set of fields makes their work harder or easier. If those two new fields do get added, let me know and I can get them indexed and searchable.

Comment 9

[Gabriele Svelto [:gsvelto] (PTO)]

Adding fields might break more stuff than it fixes so if the fix on Socorro's side is easier then I'm all for it. What I might do however is clear up the documentation for AdapterDeviceID and AdapterVendorID in mozilla-central so that it mentions that they're either PCI IDs (0xNNNN) or plain names.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

That makes a lot of sense to me.

I'll fix the indexing either today or next week.

Comment 11

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4799: fix bug 1508215: make graphics adapter/device keywords

Comment 12

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/dc59f8b09b21bb554c514c8733edc9d86428697f
fix bug 1508215: make graphics adapter/device keywords

This fixes the analysis for graphics adapter id and graphics device id
so the values are not tokenized into smaller parts. Firefox sends
values like 0x8000 which tokenizes into a single token. Fennec, however,
sends values like "Imagination Technologies" which gets broken into two
tokens and then we end up with two facets which messes up the Graphics
Adapter section in the signature summary report.

This fixes that.

https://github.com/mozilla-services/socorro/commit/bf757cad52271d427ba2d7544d5654f98fea3b44
Merge pull request #4799 from willkg/1508215-graphics

fix bug 1508215: make graphics adapter/device keywords

Comment 13

[Will Kahn-Greene [:willkg] ET needinfo? me]

Reopening this. I need to verify it on stage after stage creates a new index this weekend. Then assuming all is well, I'll push it out next week to prod.

Comment 14

[Will Kahn-Greene [:willkg] ET needinfo? me]

Looks good on stage!

Comment 15

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pushed to prod just now. We won't see any differences until Socorro creates a new ES index on Sunday. Crash reports indexed after that will show up correctly in the graphics adapter/device facets.

Marking as FIXED.