Closed Bug 1362949 Opened 3 years ago Closed 3 years ago

Build bustage for artifact builds since merge commit c3e5497cff1c: "no built artifacts found"

Categories

(Firefox Build System :: General, defect)

55 Branch
defect
Not set

Tracking

(firefox55 fixed)

RESOLVED FIXED
mozilla55
Tracking Status
firefox55 --- fixed

People

(Reporter: whimboo, Unassigned)

References

Details

Attachments

(2 files)

I have already noticed this Friday last week and talked to Chris, but then got it accidentally working. So no further action has been taken.

But since morning I cannot build artifact builds on mozilla-central for at least OS X anymore because of:

0:17.27 Attempting to find a pushhead containing c3e5497cff1c995821b1c9320fa71f1ef9a8c30e on mozilla-central.
 0:17.27 Attempting to find a pushhead containing c3e5497cff1c995821b1c9320fa71f1ef9a8c30e on integration/mozilla-inbound.
 0:17.27 Attempting to find a pushhead containing c3e5497cff1c995821b1c9320fa71f1ef9a8c30e on releases/mozilla-beta.
 0:18.26 Retrieving the last 50 pushheads starting with id 90002 on integration/mozilla-inbound
 0:18.26 Retrieving the last 50 pushheads starting with id 31776 on mozilla-central
 0:59.02 Tried 53 pushheads, no built artifacts found.
 0:59.06 make[4]: *** [recurse_artifact] Error 1
 0:59.06 make[3]: *** [artifact] Error 2
 0:59.06 make[2]: *** [default] Error 2
 0:59.06 make[1]: *** [realbuild] Error 2
 0:59.06 make: *** [build] Error 2

This is a merge commit: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c3e5497cff1c995821b1c9320fa71f1ef9a8c30e&filter-tier=1&filter-tier=2&filter-tier=3

The merge commit done right before works for me:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=22aaf8bad4df8e5f8c976f1521c213cb37e2dff5&filter-tier=1&filter-tier=2&filter-tier=3

But it installs from a remote pushhead from May 5th, which is 5 days old now:

0:03.76 Attempting to find a pushhead containing 22aaf8bad4df8e5f8c976f1521c213cb37e2dff5 on mozilla-central.
 0:03.76 Attempting to find a pushhead containing 22aaf8bad4df8e5f8c976f1521c213cb37e2dff5 on integration/mozilla-inbound.
 0:03.76 Attempting to find a pushhead containing 22aaf8bad4df8e5f8c976f1521c213cb37e2dff5 on releases/mozilla-beta.
 0:04.81 Retrieving the last 50 pushheads starting with id 90002 on integration/mozilla-inbound
 0:04.81 Retrieving the last 50 pushheads starting with id 31775 on mozilla-central
 0:21.56 Installing from remote pushhead 33b92d9c40562dab3d7b602368c75619f1d793f7 on mozilla-central

For me it happens for debug builds while Mark is trying to build opt.
Flags: needinfo?(cmanchester)
There are two issues here: one is we stopped producing debug os x buildbot builds, so the artifact code needs to update the artifacts it looks for, and the second, which seems like a real bug, is that we appear to have stopped indexing all the relevant artifacts for buildbot os x opt builds in the taskcluster index, see for instance https://tools.taskcluster.net/task-inspector/#aVkiFaQuRIaVccK5lldcnA/0
Flags: needinfo?(cmanchester)
This looks like fallout from Bug 1350413.  The first build with that ticket has Mac OS X 10.7 opt builds that don't index any artifacts; the tc Tier 2 build that indexed target.dmg is gone:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=0b255199db9d6a6f189b89b7906f99155bde3726

Compare to the push before, where there is a tc Tier 2 build indexing target.dmg:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=6d4525b5aaa834ea4db8d8eab8087d86e34cf57a&selectedJob=96615068
My comment mid-aired with chmanchester's and gives details about his second (real bug).

wcosta, dustin: can you folks dig into the indexing and figure out how to get Mac OS X opt build artifacts back into the Task Cluster index?
Flags: needinfo?(wcosta)
Flags: needinfo?(dustin)
Comment 4 restores my ability to do debug artifact builds locally.
Comment on attachment 8865569 [details]
Bug 1362949 - Update artifact build code to accept buildbot or taskcluster style artifact names for OS X.

https://reviewboard.mozilla.org/r/137192/#review140242

Makes sense.
Attachment #8865569 - Flags: review?(nalexander) → review+
Does that patch answer the question?  Anyway, I think Wander has all the state here.
Flags: needinfo?(dustin)
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b230bcb04a19
Update artifact build code to accept buildbot or taskcluster style artifact names for OS X. r=nalexander
Is my help still needed here?
Flags: needinfo?(wcosta)
Pushed by kwierso@gmail.com:
https://hg.mozilla.org/mozilla-central/rev/e0955584782e
Update artifact build code to accept buildbot or taskcluster style artifact names for OS X. r=nalexander a=merge
https://hg.mozilla.org/mozilla-central/rev/e0955584782e
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
(In reply to Wander Lairson Costa [:wcosta] from comment #9)
> Is my help still needed here?

I think so: chmanchester's patch restores Mac OS X debug builds.  But we don't have any Mac OS X opt builds in the Task Cluster index at this time (that Chris or I can find).  So we need you to either find them for us or get them re-instated in the index, ASAP.
Flags: needinfo?(wcosta)
This is only half fixed. Reopening to avoid confusion.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
The macosx64-opt build task executes through buildbot-bridge, and there
is a specific task in which artifacts are uploaded.
Flags: needinfo?(wcosta)
Attachment #8865692 - Flags: review?(dustin)
https://hg.mozilla.org/mozilla-central/rev/b230bcb04a19
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #8865692 - Flags: review?(dustin) → review+
Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/mozilla-central/rev/b9bd73036a92
Fix macosx64-opt index routes. r=dustin a=bustage-fix
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.