1038673 - Add a consistency check on the ship-it build

Status: RESOLVED WONTFIX

(Release Engineering :: Applications: Shipit, defect, P3)

Description

[Sylvestre Ledru [:Sylvestre]]

Attachment: 0001-Display-a-warning-when-a-beta-build-is-started-from-.patch

Yesterday, I made the mistake to build fennec beta 10 from the release branch.

The patch adds a check to display a warning in this case
(and could be extended to add further checks).

Comment 1

[Sylvestre Ledru [:Sylvestre]]

Attachment: Example of the result

Comment 2

[[e-mail] (:bhearsum)]

This seems like the sort of thing that release sanity should be checking, not Ship It.

When bug 847424 is fixed, this (and all the other errors that release sanity can detect) will be raised to Ship It's interface.

Comment 3

[Sylvestre Ledru [:Sylvestre]]

I am not sure I agree here. We, release managers, want to have this feedback in the ship-it interface in order to fix the potential mistake before the build starts.

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Sylvestre Ledru [:sylvestre] from comment #3)
> I am not sure I agree here. We, release managers, want to have this feedback
> in the ship-it interface in order to fix the potential mistake before the
> build starts.

This is what release sanity is for - which runs prior to the release starting, and halts the entire process if there's an error (it resets the "ready" flag in ship it, too). It's unfortunate that it doesn't expose its entire output on Ship It, but this is exactly the type of thing that release sanity is for.

Comment 5

[[e-mail] (:bhearsum)]

Comment on attachment 8456122 [details] [diff] [review]
0001-Display-a-warning-when-a-beta-build-is-started-from-.patch

Sylvestre and I talked about this on Vidyo. He's going to look at adding this as an additional check to release sanity (https://github.com/mozilla/build-tools/blob/master/buildbot-helpers/release_sanity.py). bug 847424 is tracking exposing failures from release sanity on ship it - which will make them more visible to everyone.

Comment 6

[Lawrence Mandel [:lmandel] (use needinfo)]

I agree that this should be part of the existing release sanity checks. I think there is also a point to be made to have these checks in the UI at the time when we're creating a new release. The earlier we identify potential issues the cheaper it is to fix them.

Comment 7

[[e-mail] (:bhearsum)]

Mass component change for ship it bugs.

Comment 8

[Sylvestre Ledru [:Sylvestre]]

With what we have currently in ship-it, we don't need to implement that now.