1593820 - Investigate adding a --pernosco/--rr flag to |mach try|

Status: RESOLVED FIXED

(Developer Infrastructure :: Try, enhancement)

Comment 1

[Andrew Halberstadt [:ahal]]

Hey Kyle,

I'd like to implement some sort of mach try --pernosco flag that developers can use to get Pernosco to analyze that specific push. This will require an update to the Pernosco infrastructure. My understanding is that you're listening to a pulse exchange for new pushes (or tasks?) and then comparing the author to the developer whitelist you're maintaining somewhere.

What pulse exchange are you using? Hopefully I can get --pernosco to stuff a flag in there so the update on your end would be minimal.
Thanks!

Comment 2

[Kyle Huey [:khuey]]

We listen to exchange/taskcluster-queue/v1/{task-completed,task-defined,task-exception,task-failed,task-running}.

That said, we have to download the taskcluster JSON description of the task anyways to decide if it's interesting, so stuffing it on there somewhere would be fine (and probably preferable tbh) to putting it on pulse somewhere.

We currently search for "nopernosco" in the TRY_COMMIT_MSG env variable and skip the task if it has that. Stuffing a pernosco boolean onto the task JSON somewhere that's either true (process it), false (don't process it) or missing (apply our current heuristics) would be good, I think.

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1593820 - [try] Create a ./mach try --pernosco flag to opt-in to the Pernosco debugging service, r?jmaher

Comment 4

[Andrew Halberstadt [:ahal]]

This patch stuffs a PERNOSCO env into each task, it can be "1" (explicitly requested), "0" (explicitly not requested) or undefined (use your existing heuristics). If that sounds good to you I'll land it once the infrastructure is set up to support it on your side.

This patch is simplistic and will set this PERNOSCO env on all tasks regardless of whether or not they are actually relevant to you. But if it helps, I could make sure it only applies to Linux x86_64 test tasks for example. Then you could use it to help filter out things that are irrelevant (though we'd need to make it a tri-state).

Comment 5

[Kyle Huey [:khuey]]

That sounds fine. There's no need to set it only on certain tasks, we already have to do filtering there (and more than just Linux/x86_64, we exclude certain job types that have useless debuginfo like ASAN and ccov builds and tests too).

Also I don't think you need to wait for us to land it, just to announce it. The changes required on my end are pretty trivial.

Comment 6

[Kyle Huey [:khuey]]

One thing that hasn't come up yet is that this would let anybody who can push to try trigger Pernosco sessions. But the current support for actually logging into Pernosco is limited to people with @mozilla.com email addresses ...

Comment 7

[Andrew Halberstadt [:ahal]]

Good point, maybe I can try and detect that and error out/prompt early if that's the case.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/be6ff7053ebe
[try] Create a ./mach try --pernosco flag to opt-in to the Pernosco debugging service, r=jmaher

Comment 9

[Andrew Halberstadt [:ahal]]

Hey Kyle, the flag should be on central soon. Let me know when you've made the changes on your end, then we can test and advertise.

Comment 10

[Kyle Huey [:khuey]]

I've deployed the changes on our side. To test this you need to introduce a deterministic failure in a test that runs on x86-64 Linux debug, and then push with and without the Pernosco flag. It may help if you introduce different failures in the two pushes so it's easier to tell them apart, though I think the emails you will get from the system include the hg rev of the push.

Let me know if you need any help from me.

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/be6ff7053ebe