1492128 - Send email when release build started

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, enhancement, P3)

Description

[Rail Aliiev [:rail]]

After switching to ship-it v2, we stopped sending the "build started" emails - no more releaserunner \o/. Some people requested this back, because it's useful.

It should be easy to add this as a separate task.

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

Just sending the email is something similar to release-notify-promote or so, with minor adjustements. 

For posterity whenever we'll tackle the buglist for QA as well:

jcristau> fwiw the bit QA is most interested in having back is the bug list from the old ship-it emails
16:13:20 <~mtabara> re: the bug list, that can be done too. we can move the logic from release runner  to in-tree task or something
16:17:35 <~mtabara> we have it in tools here - https://hg.mozilla.org/build/tools/file/tip/lib/python/kickoff/__init__.py#l88 and it's processed here https://hg.mozilla.org/build/tools/file/tip/lib/python/kickoff/buglist_creator.py
16:17:59 <~mtabara> not sure what/if any credentials are used, but if they are not, we could move that logic to in-tree
16:18:20 <~mtabara> that'd be even better actually but as rail said, not trivial at first glance

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Long-term solution here is to have:
* the TC notify task in order to announce the release has started
* have this run on the docker-worker so that it also runs a script
* move logic from tools repo in the docker worker and the results be pasted in the artifacts or somewhere where it can be read
* ideally the ULR of the artifacts or w\e consists the subject of the message
* we'd need a bugzilla account + secrets to access the API to grab the information we need 

Back-up solution(s) for this:
a) provide QE a way to run this command to get the bug diff
b) releaseduty to run this for QE

Comment 3

[Aki Sasaki (not active)]

We may want a "`release_promotion_flavor` started" email for each graph. For instance, if we have a `The Firefox 64.0b3 ship graph has been scheduled` email, that would alert people that it's scheduled, even if it won't run til after the promote or push graphs. It would also help us know if we accidentally hit `ship` when we didn't mean to.

(This might be a separate bug; a single email with bug info at the start of the promote phase would probably solve comment 0.)

Comment 4

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: Vendor pathlib2==2.3.2; r?build

Comment 5

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: Vendor taskcluster==4.0.1; r?build

Comment 6

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Add mozrelease tests to taskcluster; r?mtabara

Comment 7

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: Add `VOLUME` definitions to base debian images; r?glandium

For tasks that only need python, the base images are sufficient. The in-tree
code verifies that the caches are setup as volumes in the image being used,
so set the caches in the base images as well.

Comment 8

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: Set TASKCLUSTER_ROOT_URL in tasks that use the taskcluster proxy; r?dustin

Comment 9

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Import buglist_creator from build-tools; r?mtabara

This is imported from https://hg.mozilla.org/build/tools/file/482c0f6537d47f7cbf1326066ecd1d3efbbd53a7

Comment 10

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Add buglist_creator to test-suite; r?mtabara

Comment 11

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Fix flake8 errors in buglist_creator; r?mtabara

Comment 12

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Cleanup buglist_creator code; r?mtabara

Comment 13

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Add command to generate buglist for a given release; r?mtabara

Comment 14

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Send an email when a release starts with a bug list; r?mtabara

Comment 15

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Pass repository to buglist commands; r?mtabara

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1319783f4809
Vendor pathlib2==2.3.2; r=firefox-build-system-reviewers,gps
https://hg.mozilla.org/integration/autoland/rev/b314c0a4d03f
Vendor taskcluster==4.0.1; r=firefox-build-system-reviewers,gps
https://hg.mozilla.org/integration/autoland/rev/2d15a1d91cb2
[mozrelease] Add mozrelease tests to taskcluster; r=mtabara

Comment 17

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/1319783f4809
https://hg.mozilla.org/mozilla-central/rev/b314c0a4d03f
https://hg.mozilla.org/mozilla-central/rev/2d15a1d91cb2

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/778d969a1dab
Add `VOLUME` definitions to base debian images; r=gps
https://hg.mozilla.org/integration/autoland/rev/31237fb789a2
Set TASKCLUSTER_ROOT_URL in tasks that use the taskcluster proxy; r=dustin
https://hg.mozilla.org/integration/autoland/rev/bca63676be6e
[mozrelease] Import buglist_creator from build-tools; r=mtabara
https://hg.mozilla.org/integration/autoland/rev/d7a7068b519f
[mozrelease] Add buglist_creator to test-suite; r=mtabara
https://hg.mozilla.org/integration/autoland/rev/c9194dc8a470
[mozrelease] Fix flake8 errors in buglist_creator; r=mtabara
https://hg.mozilla.org/integration/autoland/rev/8f2c67323ea3
[mozrelease] Cleanup buglist_creator code; r=mtabara
https://hg.mozilla.org/integration/autoland/rev/5a4d509ac7a5
[mozrelease] Add command to generate buglist for a given release; r=mtabara
https://hg.mozilla.org/integration/autoland/rev/5e9ec632aff4
[mozrelease] Send an email when a release starts with a bug list; r=mtabara
https://hg.mozilla.org/integration/autoland/rev/b67bb5ef663e
[mozrelease] Pass repository to buglist commands; r=mtabara

Comment 19

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/778d969a1dab
https://hg.mozilla.org/mozilla-central/rev/31237fb789a2
https://hg.mozilla.org/mozilla-central/rev/bca63676be6e
https://hg.mozilla.org/mozilla-central/rev/d7a7068b519f
https://hg.mozilla.org/mozilla-central/rev/c9194dc8a470
https://hg.mozilla.org/mozilla-central/rev/8f2c67323ea3
https://hg.mozilla.org/mozilla-central/rev/5a4d509ac7a5
https://hg.mozilla.org/mozilla-central/rev/5e9ec632aff4
https://hg.mozilla.org/mozilla-central/rev/b67bb5ef663e

Comment 20

[Mihai Tabara [:mtabara]⌚️GMT]

@tom: are we uplifting this stuff to beta to immediate benefit or do should we let this ride the train?
Again, great great job with this.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/005931f03bee
[mozrelease] Fix typo in taskcluster proxy configuration; r=me

Comment 22

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/005931f03bee

Comment 23

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-beta/rev/244a824f3bf1
https://hg.mozilla.org/releases/mozilla-beta/rev/e13482c97a0f
https://hg.mozilla.org/releases/mozilla-beta/rev/2db0ba969340
https://hg.mozilla.org/releases/mozilla-beta/rev/c3105c9e0647
https://hg.mozilla.org/releases/mozilla-beta/rev/4a8f26b29962
https://hg.mozilla.org/releases/mozilla-beta/rev/997395845541
https://hg.mozilla.org/releases/mozilla-beta/rev/99694ca9da4d
https://hg.mozilla.org/releases/mozilla-beta/rev/a63fa7222a0f
https://hg.mozilla.org/releases/mozilla-beta/rev/ca613265746b
https://hg.mozilla.org/releases/mozilla-beta/rev/824771ceb6f1
https://hg.mozilla.org/releases/mozilla-beta/rev/45948f09c167
https://hg.mozilla.org/releases/mozilla-beta/rev/c00f4ecec6e7

Comment 24

[Bogdan Maris, Desktop QA]

Hello! If possible would it be too much of a hassle to change the shortURL for bugzilla queries to something more like a hyperlink with the full url? 
We have restrictions here (in the Softvision office) with shorturls, and have problems opening links like https://mzl.la/, bit.ly and others. If it’s too much to ask that's fine, we can always use some online “expand url” tools.

Other then that, it's good to see this email back in action \o/

Comment 25

[Tom Prince [:tomprince]]

(In reply to Bogdan Maris [:bogdan_maris], Release Desktop QA from comment #24)
> Hello! If possible would it be too much of a hassle to change the shortURL
> for bugzilla queries to something more like a hyperlink with the full url? 

Sure. Now that the code is in tree, it is easy to iterate on. Feel free to file other bugs for other improvements.

Comment 26

[Tom Prince [:tomprince]]

Attachment: Bug 1492128: [mozrelease] Tweak buglist email to be nicely formated and not use URL shortener; r?mtabara

Comment 27

[Tom Prince [:tomprince]]

:bogdan_maris I've sent you a sample email generated with the updated code from the above patch. If that looks good, and the code gets review, we should be able to land it for the next beta.

Comment 28

[Bogdan Maris, Desktop QA]

(In reply to Tom Prince [:tomprince] from comment #27)
> :bogdan_maris I've sent you a sample email generated with the updated code
> from the above patch. If that looks good, and the code gets review, we
> should be able to land it for the next beta.

Thanks very much, it looks brilliant from our point of view!

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7ec7580e43e4
[mozrelease] Tweak buglist email to be nicely formated and not use URL shortener; r=mtabara

Comment 30

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-beta/rev/5f5ccfe85631

Comment 31

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/7ec7580e43e4

Comment 32

[Nick Thomas [:nthomas] (UTC+12)]

I noticed we didn't get an email when Firefox 60.4.0 build2 started, is this easily upliftable to esr60 ?

Comment 33

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/6248ace3a501
https://hg.mozilla.org/releases/mozilla-esr60/rev/53fd9b608161
https://hg.mozilla.org/releases/mozilla-esr60/rev/376d2bf874d4
https://hg.mozilla.org/releases/mozilla-esr60/rev/ae67721d8d14
https://hg.mozilla.org/releases/mozilla-esr60/rev/4585f0bba945
https://hg.mozilla.org/releases/mozilla-esr60/rev/144b86ee4a5c
https://hg.mozilla.org/releases/mozilla-esr60/rev/d3cbc4397cf6
https://hg.mozilla.org/releases/mozilla-esr60/rev/0b195464716b
https://hg.mozilla.org/releases/mozilla-esr60/rev/2e17368b47c9
https://hg.mozilla.org/releases/mozilla-esr60/rev/1a026a44500b
https://hg.mozilla.org/releases/mozilla-esr60/rev/0ad7b5576268
https://hg.mozilla.org/releases/mozilla-esr60/rev/3832eeb1e124
https://hg.mozilla.org/releases/mozilla-esr60/rev/ce2b8fd01f4c
https://hg.mozilla.org/releases/mozilla-esr60/rev/27d80d202f78