buildapi doesn't differentiate between a coalesced job and one that actually completed on that revision

RESOLVED WONTFIX

Status

Release Engineering
General
RESOLVED WONTFIX
7 years ago
10 months ago

People

(Reporter: armenzg, Unassigned)

Tracking

({buildapi, sheriffing-untriaged})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2828] )

(Reporter)

Description

7 years ago
These days we are lacking some Win32 IX buildpool slaves and changes sometimes get merged.

There was a talos regression and a change was suspicious to have caused it. The problem comes that the developer was scratching his head trying to figure why there was no talos even though self-serve showed a WINNT 5.2 build. What he couldn't tell is that the build got merged with another 2 and it only showed up for the last change of the 3 of them.

Could we add a note on self-serve for jobs that got run with other changes?
Something like "(merged with cset 1234af)"

glandium: Why aren't there any windows builds on https://build.mozilla.org/buildapi/self-serve/mozilla-central/rev/d3e15e7073f9 ?
[10:59am] glandium: actually, there are builds, no tests
[10:59am] bhearsum: looks like your builds are still in progress
[10:59am] bhearsum: tests can't start until they're (mostly) done
[11:00am] glandium: they're done, according to self-serve
[11:00am] bhearsum: oh, heh
[11:00am] bhearsum: i always misread that
[11:01am] glandium: would it be possible to trigger tests (most notably talos) on this rev ?
[11:02am] bhearsum: maybe - i'll have to redirect you to armenzg_buildduty, though
[11:02am] armenzg_buildduty: glandium: let me check
[11:09am] armenzg_buildduty: glandium: it seems like 3 changes got merged ac653960b153fc8f96be4277e5edaae88461b123, d3e15e7073f92cd43479d5809b337d8ba031221d and d1228b8cedc6330a9af1ed54ebf0919c1f251e7c
[11:09am] armenzg_buildduty: glandium: that means that http://tbpl.mozilla.org/?tree=Firefox&noignore=1&rev=d1228b8cedc6330a9af1ed54ebf0919c1f251e7c has your results
[11:10am] armenzg_buildduty: I added more win32 IX machines so it should not skip
[11:10am] armenzg_buildduty: and more to come
[11:10am] glandium: armenzg_buildduty: except i want to be sure which one of d1228b8cedc6330a9af1ed54ebf0919c1f251e7c and d3e15e7073f92cd43479d5809b337d8ba031221d is responsible for the talos regression on xp
[11:10am] armenzg_buildduty: glandium: I don't have 3 builds
[11:11am] armenzg_buildduty: the best way is to push the first and second one to the try server
[11:11am] armenzg_buildduty: the wait times are null on the try server for win32 so it should be fast
[11:11am] glandium: armenzg_buildduty: supposedly, both d1228b8cedc6330a9af1ed54ebf0919c1f251e7c and d3e15e7073f92cd43479d5809b337d8ba031221d did build. at least that's what self-serve reports
[11:12am] glandium: armenzg_buildduty: the wait time on try is the time it takes to build. usually 3 hours
[11:12am] glandium: i'll be off way before that
[11:12am] armenzg_buildduty: I was meaning that slaves are picking pushes almost immediately
[11:12am] armenzg_buildduty: self-serve says that it got built
[11:12am] armenzg_buildduty: but it doesn't say that it got merge with other 2 changes
[11:13am] armenzg_buildduty: I will file a bug to see how the reporting could be changed
[11:14am] jlebar joined the chat room.
[11:14am] armenzg_buildduty: if I could help you I would
[11:14am] glandium: yeah, there's only d1228b8cedc6330a9af1ed54ebf0919c1f251e7c in http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/ 
[11:14am] armenzg_buildduty: but the only thing I could do is push the changes on your behalf, file the bug about changing how it reports and work on adding more win32 slaves
[11:14am] armenzg_buildduty: I am sorry it hit you but that is as far as I can do
[11:15am] armenzg_buildduty: I hope it makes sense

Updated

6 years ago
Component: Release Engineering → Release Engineering: Developer Tools
Priority: P3 → --
QA Contact: release → lsblakk
Hardware: x86 → All
Whiteboard: [buildapi]
QA Contact: lsblakk → hwine

Updated

5 years ago
Keywords: buildapi
Whiteboard: [buildapi]
(Assignee)

Updated

5 years ago
Product: mozilla.org → Release Engineering

Comment 1

4 years ago
In this bug, we should ensure that the coalesced jobs still show up in self-serve (so we can hit retrigger to force the job to be run on that push), but make it clear that they didn't actually run (different colour maybe?).

More context:

10:26:59 - Gijs: edmorley: so I'm looking at https://secure.pub.build.mozilla.org/buildapi/self-serve/holly/rev/fe4ac14772f1
10:27:16 - Gijs: edmorley: it has an orange Windows 7 32-bit holly debug test mochitest-other
10:27:34 - Gijs: edmorley: is that just the coalesced variant that ran against a later rev and is the rev field simply lying?
10:27:41 - edmorley: Gijs: yes
10:27:50 - edmorley: Gijs: but that is also what conventiently lets us cheat 
10:27:51 - Gijs: :(
10:27:55 - Gijs: well right
10:27:56 - edmorley: Gijs: and yeah non-obvious lol
10:28:03 - Gijs: but lying isn't nice.

by cheat I was referring to:

10:22:28 - edmorley: Gijs: you can sometimes also cheat - jobs sometimes show up in self-serve for a push, even if that push didn't run them (ie the coalesced entry shows up, but shows the result of whenever it finally ran)
10:23:09 - edmorley: Gijs: so if that push had builds for that platform (but just not tests), you can press the retrigger on the self-serve page and get the test to be scheduled without needing to wait for new builds
Keywords: sheriffing-untriaged
Summary: buildapi reports a build done for a changeset when it really got built with other two changes → buildapi reports doesn't differentiate between a coalesced job and one that actually completed on that revision

Updated

4 years ago
Summary: buildapi reports doesn't differentiate between a coalesced job and one that actually completed on that revision → buildapi doesn't differentiate between a coalesced job and one that actually completed on that revision

Comment 2

4 years ago
And again :-)

jmaher: how do I trigger a job that was coalesced
edmorley: try self-serve - the job may appear for that revision, even if it didn't run
edmorley: and thus you can request a "retrigger" of the non-existent job (as long as builds existed)
edmorley: but the retrigger will actually run on that rev, rather than the one it actually ran on
edmorley: the first time around
jmaher: odd, I see it in the self server with 'success', but it never shows up on tbpl
jmaher: is that normal for coalesced jobs?
edmorley: that's the coalesced job
edmorley: note it didn't actually run on that rev
jmaher: err
jmaher: got it
edmorley: :-)
jmaher: it would be nice if it was limey green or something

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2819]

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2819] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2827]

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2827] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2828]
Probably won't be fixing this because self serve is going to be deprecated after all scheduling moves to Taskcluster.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
(Assignee)

Updated

10 months ago
Component: Tools → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.