1432489 - test_TelemetrySend.js is going to permafail when the Gecko version number is bumped to 61 on 2018-03-12

Status: VERIFIED FIXED

(Toolkit :: Telemetry, enhancement)

Description

[Ryan VanderMeulen [:RyanVM]]

https://treeherder.mozilla.org/logviewer.html#?job_id=157875355&repo=try

The test depends on the TELEMETRY_SEND_FAILURE_TYPE telemetry probe, which is due to expire in 61. Chris, do we still need this probe? If so, please bump the expiration.

Comment 1

[Chris H-C :chutten]

That's an interesting question.

I would like to be notified if TELEMETRY_SEND_FAILURE_TYPE were to radically change in the future. But we're duplicating this reporting in the health ping.

(( Though the health ping isn't told about eTooLate[1] (introduced in bug 1397293), so maybe it isn't 100% duplicated effort ))

Alessio, Georg, what do you think about extending the expiry of TELEMETRY_SEND_FAILURE_TYPE? (and adding eTooLate to the health ping. I think I may have left it out of the health ping on purpose, but I can find no documentation of any reasoning for that.)

[1]: https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/TelemetrySend.jsm#1101

Comment 2

[Alessio Placitelli [:Dexter]]

(In reply to Chris H-C :chutten from comment #1)
> That's an interesting question.
> 
> I would like to be notified if TELEMETRY_SEND_FAILURE_TYPE were to radically
> change in the future. But we're duplicating this reporting in the health
> ping.
> 
> (( Though the health ping isn't told about eTooLate[1] (introduced in bug
> 1397293), so maybe it isn't 100% duplicated effort ))

I think it's a good idea to extend the probe (low hanging fruit) until we make sure to have the missing bit of information in the health ping. This is useful as we have alerts on this and tooling (TMO?).

> Alessio, Georg, what do you think about extending the expiry of
> TELEMETRY_SEND_FAILURE_TYPE? (and adding eTooLate to the health ping. I
> think I may have left it out of the health ping on purpose, but I can find
> no documentation of any reasoning for that.)

Could that be because it would need to be on a ping on the next startup?

Comment 3

[Ryan VanderMeulen [:RyanVM]]

TELEMETRY_SUCCESSFUL_SEND_PINGS_SIZE_KB and TELEMETRY_FAILED_SEND_PINGS_SIZE_KB also need bumping to make this test happy.

Comment 4

[Georg Fritzsche [:gfritzsche]]

(In reply to Chris H-C :chutten from comment #1)
> I would like to be notified if TELEMETRY_SEND_FAILURE_TYPE were to radically
> change in the future. But we're duplicating this reporting in the health
> ping.
> 
> (( Though the health ping isn't told about eTooLate[1] (introduced in bug
> 1397293), so maybe it isn't 100% duplicated effort ))
> 
> Alessio, Georg, what do you think about extending the expiry of
> TELEMETRY_SEND_FAILURE_TYPE? (and adding eTooLate to the health ping. I
> think I may have left it out of the health ping on purpose, but I can find
> no documentation of any reasoning for that.)

If there is a use for it, sure.
Duplication isn't a big problem i think, the value we get out of it is more important (like automatic alerting).

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Attachment: extend the expirations to 66

This extends the relevant probes to 66. Confirmed green on Try. If you're OK with this, Georg, I'll get a data-review too.

Comment 6

[Alessio Placitelli [:Dexter]]

Comment on attachment 8945146 [details] [diff] [review]
extend the expirations to 66

Looks good to me. We still need the data review for extending this I think.

@francois, data-review? on these probe extensions from version 61 to version 66. We're monitoring these to keep an eye on the Telemetry system health.

Comment 7

[François Marier [:francois]]

datareview+

Please add your own email address (or someone else) to the list of alert emails though. We now ask that there be a person next to each probe.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Which email address should I add to these before landing?

Comment 9

[Alessio Placitelli [:Dexter]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #8)
> Which email address should I add to these before landing?

Let's add [e-mail]

(In reply to Chris H-C :chutten from comment #1)
> That's an interesting question.
> 
> I would like to be notified [...]

I think he won this :-)

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e58fbaa2abbb
Extend the lifetime of the some Telemetry probes to Firefox 66. r=Dexter, data-r=francois

Comment 11

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/e58fbaa2abbb