1626020 - SchemaNotFoundException: No such docType: default-browser-agent/1

Status: RESOLVED FIXED

(Toolkit :: Default Browser Agent, defect, P1)

Description

[Anna Scholtz [:ascholtz]]

The structured schema errors dashboard show that the default-browser-agent namespace contains pings that are not being ingested since the tables do not exist.

Comment 1

[Anna Scholtz [:ascholtz]]

I did some investigation:
https://console.cloud.google.com/bigquery?sq=1018637472215:a4722e3dfb3b4be4ab0469fb91b479eb

The pings get sent by pingsender so it seems that a new ping got added but a schema for it doesn't exist yet.

Comment 2

[Anna Scholtz [:ascholtz]]

This got added recently as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1602463

In the code: https://searchfox.org/mozilla-central/source/toolkit/mozapps/defaultagent/Telemetry.cpp#28

Comment 3

[Mark Reid [:mreid]]

Is this covered by the schema added in this PR?
https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/495/files

Comment 4

[Anna Scholtz [:ascholtz]]

Indeed. The problem here is that submission URIs are incorrect. Currently, their format is /submit/default-browser-agent/1/default-browser/<ID> while the decoder expects /submit/default-browser-agent/default-browser/1/<ID> .

Comment 5

[Anna Scholtz [:ascholtz]]

In https://searchfox.org/mozilla-central/source/toolkit/mozapps/defaultagent/Telemetry.cpp#33 the line needs to be changed to:

#define TELEMETRY_PING_URL
  TELEMETRY_BASE_URL "/" TELEMETRY_NAMESPACE "/" TELEMETRY_PING_DOCTYPE
                     "/" TELEMETRY_PING_VERSION "/"

Comment 6

[Molly Howell (she/her) (mostly inactive)]

.... oh good grief. I swear I've done this before. Okay. Thank you very much for hunting that down; since it's my fault, I will take it from here.

Comment 7

[Molly Howell (she/her) (mostly inactive)]

Bug 1602463 was uplifted to 75, so everything down to that is affected.

Comment 8

[Molly Howell (she/her) (mostly inactive)]

Attachment: Bug 1626020 - Fix default browser agent telemetry URL. r=bytesized

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e911977b9bfc
Fix default browser agent telemetry URL. r=bytesized

Comment 10

[Molly Howell (she/her) (mostly inactive)]

Attachment: Bug 1626020 followup - fix a default browser ping field name to match the schema. r=bytesized

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9d1cf3c852ae
followup - fix a default browser ping field name to match the schema. r=bytesized

Comment 12

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/e911977b9bfc
https://hg.mozilla.org/mozilla-central/rev/9d1cf3c852ae

Comment 13

[Julien Cristau [:jcristau]]

Is this ready for beta uplift?

Comment 14

[Molly Howell (she/her) (mostly inactive)]

It should be, I'll request it now. I was kind of waiting for confirmation that we wanted to do that (normally probably wouldn't but this thing's been such a debacle).

Comment 15

[Molly Howell (she/her) (mostly inactive)]

Comment on attachment 9138561 [details]
Bug 1626020 - Fix default browser agent telemetry URL. r=bytesized

Beta/Release Uplift Approval Request

• User impact if declined: Default browser agent telemetry appears to send successfully, but then can't be received into the pipeline, so we get no data
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a minor change to fix the order of components in the default browser ping submission URL and fix the name of one field in the ping; there are no changes to the data itself, only to how it's submitted.
• String changes made/needed:

Comment 16

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9138561 [details]
Bug 1626020 - Fix default browser agent telemetry URL. r=bytesized

Fixes the default browser agent telemetry pings so they can be properly processed. Approved for 76.0b3.

Comment 17

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/releases/mozilla-beta/rev/8073714eef8e
https://hg.mozilla.org/releases/mozilla-beta/rev/9c95dee15490

Comment 18

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/gcp-ingestion/pull/1268

Comment 19

[Wil Stuckey]

Since this has been fixed for a couple years now, I have switched the exception class to UnwantedDataException. That will prevent these errors from showing up in our monitoring. See: https://github.com/mozilla/gcp-ingestion/pull/1998