1186898 - Turn off buildbot Flame-KK builds on trunk

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

We've been running Flame-KK builds on Taskcluster for awhile now on trunk. As far as I know, they're working OK. To save resources, we'd like to turn off the buildbot-based builds at this point.

Naoki, wcosta said to ask you about this before proceeding. Does that sound OK to you? This is trunk-only for now. We'll sort out the branches later.

Comment 1

[Wander Lairson Costa]

Bug 1174336 is supposed to add v2.1 and v2.2 to TC, but it feels like these builds will be gone soon.

Comment 2

[Wander Lairson Costa]

One more note: afaik, TC doesn't support nightly builds yet, only per commit builds.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

(In reply to Wander Lairson Costa [:wcosta] from comment #2)
> One more note: afaik, TC doesn't support nightly builds yet, only per commit
> builds.

And that's going to make life way more difficult :(. Killing the dep builds is pretty easy, but not while leaving the nightlies intact. In fact, we had a similar situation as this when killing emulator builds on trunk as they transitioned to TC and apparently decided to accept the loss of nightlies along the way.

Wander, how easily could we create Flame-KK dogfooding builds on m-c like we have for Aries? My understanding is that those are effectively nightlies?

Comment 4

[Wander Lairson Costa]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #3)
> (In reply to Wander Lairson Costa [:wcosta] from comment #2)
> > One more note: afaik, TC doesn't support nightly builds yet, only per commit
> > builds.
> 
> And that's going to make life way more difficult :(. Killing the dep builds
> is pretty easy, but not while leaving the nightlies intact. In fact, we had
> a similar situation as this when killing emulator builds on trunk as they
> transitioned to TC and apparently decided to accept the loss of nightlies
> along the way.
> 
> Wander, how easily could we create Flame-KK dogfooding builds on m-c like we
> have for Aries? My understanding is that those are effectively nightlies?

Dogfood is another beast created for Spark foxfooding program. What is closest to nightly in TC is "ota" builds, which update balrog and push symbols to crash reporter. TC only indexes successful builds, so, in principle, we don't need nightlies, QA or whoever is interested in getting a successful build, can get the latest build whenever they want.

Comment 5

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Um, we need nightly builds to qualify and we need to move automation ( raptor + treeherder + QA automation) before we do this completely.

+jlorenzo, +eli

Comment 6

[Johan Lorenzo [:jlorenzo]]

The Jenkins QA server keeps downloading builds from pvtbuilds. We're working on the getting the downloads from Task Cluster. We've had the flame-kk-spark-eng build and the aries one for 3 days, but we'd like to have them running for several weeks before migrating everything to Task Cluster.

Then, even if we had nightly builds on TC, I think it's better if we postpone the shut down of mozilla-central while automation has fully migrated to Task Cluster.

Comment 7

[:Eli Perelman]

The Raptor automation relies on the download job for flame-kk, which isn't a nightly job. Currently Raptor automation doesn't do any nightly tests, so I'm not sure anything is really affected for it. As long as the current download job switches from PVT to Taskcluster I think it will be fine.

Comment 8

[Johan Lorenzo [:jlorenzo]]

Following up from the conversation I had with Ryan on IRC:

Every download job we have, currently uses pvtbuilds. The only exceptions are 2 that are on taskcluster only (flame-kk-eng and aries-eng). They were set up by the end of last week, and we're still tuning them. 

We're late on the migration because we've been able to programmatically download builds on Jenkins since last week, thanks to [1]. mozilla-download doesn't provide the support to get Firefox OS builds.

[1] https://github.com/askeing/taskcluster-util-python
[2] https://github.com/mozilla-b2g/mozilla-download

Comment 9

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

We still need Taskcluster builds for at least 2.2 for comparison and regression checking.
2.1 it's up in the air.

Comment 10

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Also to note, I think we need to change buildbot to use mozharness to create nightly task cluster builds for flame-kk and aries.  I think this will solve the issue.

What I found is that the nightly task overrides with the ftp and doesn't alias flame.  This is causing an issue with the nightly builds.  We need to revamp the script so it points to using taskcluster for both and builds it at that time for both.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Automation for 2.1 has already been shut off (and no new patches are landing on it either - so the last build available should suffice).

Note that while I'm not ruling it out entirely, currently Taskcluster only runs builds on trunk (hence why this bug was filed only caring about trunk). Everything on b2g37 is handled by buildbot. Is there a particular reason you want 2.2 builds from Taskcluster?

Comment 12

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

It turns out that we need 2.0/2.1 builds for Marketplace testing.  We would need 2.2 for the very same reason.

They do testing on production versions.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

(In reply to Naoki Hirata :nhirata (please use needinfo instead of cc) from comment #12)
> It turns out that we need 2.0/2.1 builds for Marketplace testing.  We would
> need 2.2 for the very same reason.

What's stopping them from using the last-available builds from those releases given that the branches are now closed and not receiving any new changes?