Closed Bug 1363048 Opened 7 years ago Closed 6 years ago

Add list of relevant bugs to release-drivers email

Categories

(Release Engineering :: Applications: Shipit, enhancement, P1)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: csheehan, Assigned: csheehan, Mentored)

Details

Attachments

(3 files)

Upon submitting a build through Ship-It, an email is sent to release-drivers@m.o with information about the commit, task group, locales and creator/starter. We would like to add a list of bugs handled since the previous build with some links, for reference.
Priority: -- → P1
With the changeset from shipit, you can get the list of bugs from the mercurial pushlog. For example, here is the list of changes between 53.0 release and 53.0.2 (the next desktop dot release): 

https://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=d345b657d381&tochange=f87a819106bd

You could then pull out the bug numbers, exclude anything that's a test-only change, and de-duplicate the list. Though, it might be useful to list anything with a commit message beginning with "Backout" separately. 

I guess in the email this could be a link to one bugzilla list page with just those bugs (rather than listing each bug in the email itself).
Most of the work for this is done (figuring out previous release version, getting the changesets, making the link, tests, etc). Before I make a PR I had a few questions, specifically was wondering if there is a specific format that should be used in the emails. Right now it would look something like this:

Comparing Mercurial tag FIREFOX_53_0_RELEASE to FIREFOX_53_0_2_RELEASE:
Bugs since previous changeset: https://mzl.la/2r2n2vK
Backouts since previous changeset: https://mzl.la/2r2xFi1

As a side note, this code does not compare across major versions (no 53 vs 52, etc) and does not compare beta 1 releases to anything.

Rail, the code is here in case you wanted to poke through it:

https://github.com/cgsheeh/build-tools/blob/relevant_buglist_email/lib/python/kickoff/buglist_creator.py
https://github.com/cgsheeh/build-tools/blob/relevant_buglist_email/lib/python/kickoff/test/test_buglist_creator.py
Flags: needinfo?(rail)
Flags: needinfo?(lhenry)
Looks good! Thank you very much! I think this will come in handy for anyone on the release-drivers mailing list.   I often get asked the question over and over on irc .... "what just landed" and then have to teach people how to fiddle around with the pushlog. 

We may want to leave out bugs in the Build Config and Release Automation components, because they are not often relevant for QE (and they aren't normally changes that release management needs to approve or is looking for).
 
We could also have a 2nd link to the full changeset  (in case someone has a reason to dig into test only or config changes).  

Andrei, what do you think, will this be useful for your team? Is there anything else you would like to see?
Flags: needinfo?(lhenry) → needinfo?(andrei.vaida)
It looks great! I have come comments:

1) You don't have any tags fro the "current" version, when the message is generated. We tag the repos only after we ship the release. So instead of using its tag, you should use the revision (release["revision"]).

2) As Liz mentioned above, feel free to filter out "a=release" commits. They are used by Release Engineering to uplift release automation bugs. Maybe instead of is_test_only_change() you can create some generic function to unwanted "a=xyz" messages.

3) the tests try to fetch the data from the Internet, so they would fail offline. Can you use data snapshots (dump the json responses and load them runtime) in tests instead.

Can't wait to see the first email sent to release-drivers! :)
Flags: needinfo?(rail)
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #3)
> Andrei, what do you think, will this be useful for your team? Is there
> anything else you would like to see?

I think this looks great! I don't have anything to add, looking forward to seeing this live!
Flags: needinfo?(andrei.vaida)
Comment on attachment 8867836 [details] [review]
Adds links to all bugs and backout bugs found in Hg changeset descriptionss for a given release

Let's ship it!
Attachment #8867836 - Flags: review?(rail) → review+
Comment on attachment 8867836 [details] [review]
Adds links to all bugs and backout bugs found in Hg changeset descriptionss for a given release

https://hg.mozilla.org/build/tools/rev/e4946f92cd897999f6a397e57b5dcaf38787d9ef
Attachment #8867836 - Flags: checked-in+
Mihai, can you take care and land this change?
Flags: needinfo?(mtabara)
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #9)
> Mihai, can you take care and land this change?

Certainly. I'm not sure if we can deploy this on production as we already have a few changes pushed in the tools history that, once pulled to bm85 will break unless all related changes are completed.

However, I'll have a look at this tomorrow morning, fresh brainz to double check. I could be wrong and maybe we can get this landed on production. Otherwise, it'll go to staging/dev release runner first, if that's okay with you.

I'm leaving the NI here to remind myself tomorrow to tackle this.
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #9)
> Mihai, can you take care and land this change?

fwiw - I landed this on staging environment on bm83 for the moment.
Still evaluating if it's an option to land this on production as well.
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #11)
> (In reply to Rail Aliiev [:rail] ⌚️ET from comment #9)
> > Mihai, can you take care and land this change?
> 
> fwiw - I landed this on staging environment on bm83 for the moment.
> Still evaluating if it's an option to land this on production as well.

Pulled on production by :rail.
Flags: needinfo?(mtabara)
FYI - 52.1.2esr was started a couple of hours back, after :sheehan's patch was deployed. Haven't seen though any additional information. Not sure if it's the regex excluding ESR or something to do with the fact that we had a different branch than `default` shipping the release off, namely FIREFOX_ESR_52_1_X_RELBRANCH. Just thought this might be useful. We'll have a dot release 53.0.3 coming later on today PST so we could wait to see how that behaves.
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #13)
> FYI - 52.1.2esr was started a couple of hours back, after :sheehan's patch
> was deployed. Haven't seen though any additional information. Not sure if
> it's the regex excluding ESR or something to do with the fact that we had a
> different branch than `default` shipping the release off, namely
> FIREFOX_ESR_52_1_X_RELBRANCH. Just thought this might be useful. We'll have
> a dot release 53.0.3 coming later on today PST so we could wait to see how
> that behaves.

Firefox-53.0.3-build1 was started too sans the information. I'm assuming there's some tweak/config somewhere that didn't enable this code yet.
It failed with the following message:

2017-05-18 02:38:11,966 - INFO - ('00000053', '00000000', '00000003', '*final') is not in list

At least I think, that the message is related. :)
> It failed with the following message:
> 
> 2017-05-18 02:38:11,966 - INFO - ('00000053', '00000000', '00000003',
> '*final') is not in list
> 
> At least I think, that the message is related. :)

It is related! The code works by grabbing all the Hg tags for a branch, filters for PRODUCT_*_RELEASE in the tag, turns from the tag style versioning to the usual dot-style (53.0.3 in the most recent case) and sorts all the entries based on the dot style versions. Then it looks up the version of the release currently in flight and grabs the previous version, returning the Hg tag so we can look up all the bugs.

In all my test data the current version I was testing for already had a published Hg tag in the collection, however in a real use case the tag doesn't exist yet. So when the code tried to determine the previous version by indexing the current version and decrementing, it couldn't find the current version, hence the above error.

I've added a commit to append the current version before doing the sorting, so this lookup doesn't fail.
From last beta ....

Comparing Mercurial tag FIREFOX_54_0b8_RELEASE to FIREFOX_54_0b9_RELEASE:
Bugs since previous changeset: https://mzl.la/2pYeMt6
Backouts since previous changeset: https://mzl.la/2pYvjwP
Full Mercurial changeset: https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_54_0b8_RELEASE&tochange=54f218cec92ffcc2cc8160ed80c58ffe8b6c268d&full=1

Seeeeet! \o/
Looks great!
Attachment #8869433 - Flags: review?(rail)
Attachment #8869433 - Flags: review?(rail) → review+
I pushed a little fix to broaden the exceptions, so we catch unexpected cases. In the first build of Devedition failed to fetch the pushlog (404) due to missing tag for the previous release.

https://hg.mozilla.org/build/tools/rev/ac62025e5cefa2821e64be4ca87a21fa3d819c5b
Comment on attachment 8871534 [details]
Bug 1363048 - Handle Devedition

https://reviewboard.mozilla.org/r/143004/#review146720
Attachment #8871534 - Flags: review?(aki) → review+
I believe this is done.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Component: Applications: ShipIt (backend) → Applications: ShipIt
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: