984392 - submissions to balrog shouldn't silently fail

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

I broke Fennec nightly submission to Balrog over in bug 984163 & 984166. It wasn't caught until days later because we set flunkOnFailure=False for all Balrog submissions. This is something we inherited from snippet uploads.

We should set the overall build as something other than green when Balrog submission fails, but we shouldn't do anything that would trigger a RETRY (or suggest to someone that it should be retried). Sheriffs, what do you think would be best for this? I was thinking EXCEPTION (purple), but I'm not sure whether or not that will end up with someone retriggering it most of the time.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

We retrigger purples, yes. What would be wrong with this just causing the build to go red?

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #1)
> We retrigger purples, yes. What would be wrong with this just causing the
> build to go red?

I recall that we've been asked not to use red for infra issues in the past, because it confuses with code issues. That may be very old knowledge though, and I'm happy to use FAILURE/red if it works for you!

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Signing server issues turn builds red, as do upload failures. Not sure what the difference would be.

Comment 4

[[e-mail] (:bhearsum)]

Attachment: burn it

Comment 5

[Rail Aliiev [:rail]]

in production