1657142 - High schema error rates for `environment/system/gfx/adapters/N/GPUActive`

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect, P2)

Description

[Frank Bertsch [:frank]]

This dashboard shows schema error rates. We're tracking an error in the environment at GPUActive. It looks like this really started increasing over the past few weeks, and the voice ping (which is, albeit, a very tiny pop) is disproportionately affected. However the main, event, and rest of the telemetry pings are also affected.

Comment 1

[Frank Bertsch [:frank]]

Chutten, do you know of any changes to this field that could cause this?

Comment 2

[Chris H-C :chutten]

Ohhhh, I see. It'll be bug 1651425

For setups where there are more than 2 GPUs, we now report all of them instead of just the first 2. Unfortunately the code responsible for GPUActive wasn't updated at the same time so it only supplies a value for the first two.

As a result I expect any ping coming from a Environment with more than two GPUs will be invalid (because it's missing that field).

Guess the correct person to ni? from the people responsible there would be.... jgilbert? How would you like to proceed?

Comment 3

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/596

Comment 4

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: 2020-08-27-GPUActive errors.png

Comment 5

[Anthony Miyaguchi [:amiyaguchi]]

The number of errors drops abruptly on 2020-08-20, which corresponds to the deploy date of the PR. I'll be filing a follow up bug for potential backfill.

https://sql.telemetry.mozilla.org/queries/74258#185635

Comment 6

[Markus Stange [:mstange]]

(In reply to Chris H-C :chutten from comment #2)

Ohhhh, I see. It'll be bug 1651425

It doesn't look like this patch actually landed. It's reviewed, but the bug is still open and I don't see a checkin comment.

Comment 7

[Chris H-C :chutten]

You're right. What the heck.

Lessee, what changes have there been to GPU Active? The only change more recent than 2016 was Bug 1676883 which was last November, far after the problem was first noticed.

It's a mystery.