1822139 - Missing columns in `firefox_desktop_background_update.background_update_v1` and `metrics_v1` for `metrics`.`string_list`.`background_update.reasons_to_not_update`.[...]

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Comment 1

[Chris H-C :chutten]

:nalexander, are those metrics that were added to browser/components/metrics.yaml supposed to be sent by the BUA? That metrics.yaml is only in firefox_desktop_metrics in metrics_index.py...

Comment 2

[Nick Alexander :nalexander [he/him]]

(In reply to Chris H-C :chutten from comment #1)

:nalexander, are those metrics that were added to browser/components/metrics.yaml supposed to be sent by the BUA? That metrics.yaml is only in firefox_desktop_metrics in metrics_index.py...

Hmm, yes, we wanted to add them to the BUA as well, just for consistency. But it's not the most critical; the data was already in BUA under a different name with slightly different semantics.

Comment 3

[Chris H-C :chutten]

(In reply to Nick Alexander :nalexander [he/him] from comment #2)

(In reply to Chris H-C :chutten from comment #1)

:nalexander, are those metrics that were added to browser/components/metrics.yaml supposed to be sent by the BUA? That metrics.yaml is only in firefox_desktop_metrics in metrics_index.py...

Hmm, yes, we wanted to add them to the BUA as well, just for consistency. But it's not the most critical; the data was already in BUA under a different name with slightly different semantics.

Then you'll want to add it to background_update_metrics as well. This will resolve over the following two days as fog-update-bot will propagate the change to the pipeline.

Comment 4

[Nick Alexander :nalexander [he/him]]

mpohle: can you take this one liner? Thanks!

Comment 5

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander [he/him] from comment #4)

mpohle: can you take this one liner? Thanks!

We'll want to add

browser/components/metrics.yaml

around https://searchfox.org/mozilla-central/rev/82828dba9e290914eddd294a0871533875b3a0b5/toolkit/components/glean/metrics_index.py#61.

Comment 6

[Max Christian Pohle [:/dev/max]]

Attachment: Bug 1822139 - Missing columns in metrics.yaml for BUA.reasons_to_not_update, r=nalexander

Comment 7

[Max Christian Pohle [:/dev/max]]

As announced earlier I am now coming back this topic before this week ends. As I understand this is the desired change and I assume, that this bugs title is an error or warning message of which I do not know its origin. I will now start to search for this test and look if this change fixes it and also try to find more information about the fog-update-bot in order to gain a better understanding what makes this change is necessary. I will also appreciate further comments regarding that.

Comment 8

[Chris H-C :chutten]

(In reply to Max from comment #7)

As announced earlier I am now coming back this topic before this week ends. As I understand this is the desired change and I assume, that this bugs title is an error or warning message of which I do not know its origin. I will now start to search for this test and look if this change fixes it and also try to find more information about the fog-update-bot in order to gain a better understanding what makes this change is necessary. I will also appreciate further comments regarding that.

I can help with this.

This error is a result of a health dashboard monitoring schemas in datasets inside the Data Platform. Basically, the Firefox Desktop Background Update Agent's "background-update" ping is sending a string_list metric of name background_update.reasons_to_not_update but the dataset for that ping doesn't have a column for it. Which columns go into which datasets for which senders is determined by repositories.yaml. fog-update-bot reads toolkit/components/glean/metrics_index.py and updates probe-scraper's repositories.yaml nightly. So all you need to do to is exactly what you have done: the bot and Data Engineering will take it from here.

Landing this is sufficient to resolve this bug. If it doesn't fix the problem, this error will come up again in monitoring and end up as either this bug reopened or a new one filed.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d5235b3d4e8a
Missing columns in metrics.yaml for BUA.reasons_to_not_update, r=nalexander,chutten

Comment 10

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/d5235b3d4e8a