Can't build debug on Linux64 because the artifact names changed when we switched to taskcluster

RESOLVED FIXED in Firefox 49

Status

()

Firefox
Build Config
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: Gijs, Assigned: chmanchester)

Tracking

Trunk
Firefox 49
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox49 fixed)

Details

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(3 attachments)

(Reporter)

Description

2 years ago
Looking at:

https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-central.latest.firefox

it seems we now have items for:

linux64-dbg

and

linux64-debug

and the later has not been updated for 2 weeks now, which coincides nicely (in my memory, at least) with when we switched to taskcluster for those builds.

This means that if you use a linux64 debug artifact build, you're now SOL, because this happens:

 0:15.87 Attempting to find a pushhead containing b0096c5c727749ad3e79cbdf20d2e96bd179c213 on mozilla-central.
 0:15.87 Attempting to find a pushhead containing b0096c5c727749ad3e79cbdf20d2e96bd179c213 on integration/fx-team.
 0:15.87 Attempting to find a pushhead containing b0096c5c727749ad3e79cbdf20d2e96bd179c213 on integration/mozilla-inbound.
 0:15.87 Attempting to find a pushhead containing b0096c5c727749ad3e79cbdf20d2e96bd179c213 on releases/mozilla-aurora.
 0:16.72 Retrieving the last 50 pushheads starting with id 77464 on integration/mozilla-inbound
 0:16.72 Retrieving the last 50 pushheads starting with id 19399 on integration/fx-team
 0:16.72 Retrieving the last 50 pushheads starting with id 30289 on mozilla-central


and then it just sits there and if you wait a really really really long time it'll tell you it couldn't find any usable push heads.

This is very sad because full debug builds on my VM take a very long time.

I *think* we can fix by just updating artifacts.py or whatever it is to look for the new linux64-dbg artifacts. But the filenames also changed and seem to now all be "target.something", which might further upset it. I'm not sure yet. Chris, do you know why all the names changed?
Flags: needinfo?(cmanchester)
(Reporter)

Comment 1

2 years ago
OK, so it's even worse because apparently we index into https://tools.taskcluster.net/index/artifacts/#buildbot.branches.mozilla-central/buildbot.branches.mozilla-central which as the name might suggest only has buildbot things.
(Assignee)

Comment 2

2 years ago
I think we're not supposed to be using the buildbot index anymore anyway. At any rate, I'll take care of it.
Assignee: nobody → cmanchester
Flags: needinfo?(cmanchester)
(Reporter)

Comment 3

2 years ago
Created attachment 8757029 [details] [diff] [review]
fix artifact builds to not use buildbot index
(Assignee)

Comment 4

2 years ago
According to mshal the buildbot route will include Taskcluster builds. It gives us the ability to query by revision first, which we're using, so I think we need to stick with this for now.
(Assignee)

Comment 5

2 years ago
Mike, I'm going to push a patch your way that allows us to keep using the buildbot route. If there's a relatively easy way to get this in gecko.v2, let me know.
I'm fine with continuing to use buildbot.* for now. Even if you switched to gecko.v2, it would still use 'linux32-dbg', since that is the build_name and build_type specified in the task definitions. The routes come from here: https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/routes.json

A few of the task definitions specify 'dbg', which should be changed to 'debug':

./tasks/builds/dbg_linux32.yml:    build_type: 'dbg'
./tasks/builds/dbg_macosx64.yml:    build_type: 'dbg'
./tasks/builds/dbg_linux64.yml:    build_type: 'dbg'

Other tasks in taskcluster use 'debug', and mozharness (which is responsible for setting up the routes in buildbot builds) uses 'debug'.

Additionally, it seems the buildbot builds used 'linux' as the platform name for linux32, whereas taskcluster now uses 'linux32'. We should probably change these back to just 'linux' for consistency. It might be worth double-checking the others as well - here's the list of current platform names used in mozharness:

./releng_base_android_64_builds.py:    'platform': 'android',
./releng_base_mac_64_cross_builds.py:    'platform': 'macosx64',
./releng_base_windows_32_builds.py:    'platform': 'win32',
./releng_base_linux_32_builds.py:    'platform': 'linux',
./releng_base_windows_64_builds.py:    'platform': 'win64',
./releng_base_linux_64_builds.py:    'platform': 'linux64',
./releng_base_mac_64_builds.py:    'platform': 'macosx64',
./releng_sub_linux_configs/64_stat_and_opt.py:    'platform': 'linux64',
./releng_sub_linux_configs/64_artifact.py:    'platform': 'linux64',
Yes, let's please get these fixed, even if it causes some momentary pain.  I noticed the difference a while back, but wasn't sure which was the "right" way to name the platforms.  Ideally the right names would end up in taskcluster/docs/attributes.rst.
(Assignee)

Comment 8

2 years ago
There's a related problem, which is that we appear to be sending both opt and debug linux64 builds to the same route right now. Before we turned off buildbot builds for this platform, maybe this was helping us a bit, because they wouldn't conflict with the buildbot debug builds (and I guess we're getting this effect from the naming inconsistency on other platforms).

For now I have a band-aid patch for linux.
(Assignee)

Comment 9

2 years ago
Created attachment 8757356 [details]
MozReview Request: Bug 1276037 - Update routes for linux64 debug builds to reflect that they are debug builds. r=mshal

Review commit: https://reviewboard.mozilla.org/r/55648/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/55648/
Attachment #8757356 - Flags: review?(mshal)
Attachment #8757357 - Flags: review?(mshal)
(Assignee)

Comment 10

2 years ago
Created attachment 8757357 [details]
MozReview Request: Bug 1276037 - Update artifact builds to be able to find Linux taskcluster builds. r=mshal

Review commit: https://reviewboard.mozilla.org/r/55650/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/55650/

Updated

2 years ago
Attachment #8757356 - Flags: review?(mshal) → review+
Comment on attachment 8757356 [details]
MozReview Request: Bug 1276037 - Update routes for linux64 debug builds to reflect that they are debug builds. r=mshal

https://reviewboard.mozilla.org/r/55648/#review52604

Should we do the same for macosx64 debug?

Updated

2 years ago
Attachment #8757357 - Flags: review?(mshal) → review+
Comment on attachment 8757357 [details]
MozReview Request: Bug 1276037 - Update artifact builds to be able to find Linux taskcluster builds. r=mshal

https://reviewboard.mozilla.org/r/55650/#review52614
Can we fix the gecko.v2 routes as well here? Or file a followup?
(Assignee)

Comment 14

2 years ago
https://reviewboard.mozilla.org/r/55648/#review52604

I'm not sure, it seems like this might create confusion with the existing builds uploaded to that route by buildbot.
(Assignee)

Updated

2 years ago
Blocks: 1276352
(Assignee)

Comment 15

2 years ago
(In reply to Michael Shal [:mshal] from comment #13)
> Can we fix the gecko.v2 routes as well here? Or file a followup?

File bug 1276352. Thanks for the speedy reviews!

Comment 16

2 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/00e5d65e076c
https://hg.mozilla.org/integration/mozilla-inbound/rev/e2261f30d0a7

Comment 17

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/00e5d65e076c
https://hg.mozilla.org/mozilla-central/rev/e2261f30d0a7
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox49: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 49
You need to log in before you can comment on or make changes to this bug.