Closed Bug 1227355 Opened 4 years ago Closed 4 years ago

Need taskcluster build for b2g-ota branch

Categories

(Taskcluster :: Operations and Service Requests, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla45

People

(Reporter: hwine, Assigned: garndt)

References

Details

(Whiteboard: [b2g-ota])

Attachments

(4 files)

+++ This bug was initially created as a clone of Bug #1218059 +++
Mahe,

What jobs need to be run on this branch? Same as mozilla-central? Or something else?
Component: General → Service Request
Flags: needinfo?(mpotharaju)
We want to make builds for Aries and Flames to being with.

Can you please share what all is on Central, so we can work on the necessary list.  

Want to think through if Nexus should on this list.
Flags: needinfo?(mpotharaju)
Flags: needinfo?(sdeckelmann)
List of jobs for mozilla-central running:

mach taskcluster-graph --project mozilla-central  --message "useful message" --head-repository http://hg.mozilla.org/mozilla-central --head-rev tip --owner garndt@mozilla.com --print-names-only
This is a much more detailed look at the graph that's generated for a mozilla-central push.  It's a graph of all the tasks created.
I've attached two textual outputs from our command that generates a graph to run against mozilla-central.  Also, you can view any one of our decision tasks[1] that run on m-c.  Here is one example:

https://tools.taskcluster.net/task-graph-inspector/#HHz6BuyXR_W9gOP2-tturA/

Some of this is a little difficult to sift through because of how many tasks there are, but ping me and I can help understand what's out there and what you would like to schedule.  We have multiple versions of aries and flame builds that are done for mozilla-central so please double check this by either reviewing what I attached or looking at a set of jobs on treeherder to determine if that is also what you would like.

Also, here are a couple of links that are the configurations for mozilla-central to understand what is being looked at to build the graph of jobs that get run.

This might be too much information, but if you ever wonder what files determine what gets included, this [1] is the entry point which then inherits from [2] to build a complete graph of tasks.  Everything lives under <gecko>/testing/taskcluster/tasks/

[1] decision tasks are what decides the tasks that should be submitted for a given push.  The result is usually the graph json I attached.
[2] https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/branches/mozilla-central/job_flags.yml
[3] https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/branches/base_jobs.yml
Could we just utilize the dogfood build that's already existing, and modify it so that it updates a release on balrog and uses the OTA branch instead of creating a new build/new index?
https://docs.google.com/spreadsheets/d/1RJgs-AkubSB3I1HgRZ6n7if0UC3bE4D1rXo3WR_ZxSU/edit?pli=1#gid=2117446174&vpid=A30

lists all the builds that are on MC and there's some that are duplicates or unnecessary at this point in time.
(In reply to Naoki Hirata :nhirata (please use needinfo instead of cc) from comment #6)
> Could we just utilize the dogfood build that's already existing, and modify
> it so that it updates a release on balrog and uses the OTA branch instead of
> creating a new build/new index?

All index paths I'm aware of have branchnames in them.  Greg dug up this in our configs, which indicates the branch name is part of the existing dogfood builds: https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/phone_build.yml#21

If that's not the case with the dogfood build you're referring to, can you provide a link for us? 

For now, anything new that updates AUS/Balrog is not possible. RelEng is meeting this week to discuss how we can support AUS for B2G going forward.
Flags: needinfo?(sdeckelmann)
(In reply to Naoki Hirata :nhirata (please use needinfo instead of cc) from comment #7)
> https://docs.google.com/spreadsheets/d/1RJgs-
> AkubSB3I1HgRZ6n7if0UC3bE4D1rXo3WR_ZxSU/edit?pli=1#gid=2117446174&vpid=A30
> 
> lists all the builds that are on MC and there's some that are duplicates or
> unnecessary at this point in time.

Considering comment 8, are there any m-c specific builds/tests that you would like enabled on this branch that does *not* require contacting balrog?

If so, please let me know from the list you gave what jobs on m-c you would like to see enabled and when you would like those enabled.
It appears that based on commend 2, Aries and Flame are what we're concerned with in this branch.  The list below are the flame and aries builds that appear on m-c.  Most should be able to be enabled on b2g-ota but Aries OTA (user and userdebug) currently use a mozharness config that submits to balrog, which is not possible to be enabled on this branch currently.  I'm not sure what should be done about those unless we have a config for those that does not submit to balrog but produces the updates.

- [TC] B2G Flame KK Debug
- [TC] B2G Flame KK Opt
- [TC] B2G Flame KK Eng
- [TC] B2G Flame KK Eng (spark)
- [TC] B2G Flame KK OTA (userdebug)
- [TC] B2G Flame KK OTA (user)
- [TC] B2G Aries Debug
- [TC] B2G Aries Opt
- [TC] B2G Aries OTA (userdebug)
- [TC] B2G Aries OTA (user)
- [TC] B2G Aries Eng
- [TC] B2G Aries Dogfood
 (In reply to Greg Arndt [:garndt] from comment #10)
> It appears that based on commend 2, Aries and Flame are what we're concerned
> with in this branch. 

Please let me clarify my comment 2 here. 
We want to BEGIN with Aries and Flame, but be able to provide updates to Nexus 4,5,6 and other phones in the future once we have the How, When and other related questions answered.
Bug 1227355 - Configure b2g-ota branch and add update channel r=wcosta
Attachment #8693936 - Flags: review?(wcosta)
Assignee: nobody → garndt
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

So this patch should enable aries and flame builds on b2g-ota release branch.  Also some changes were made to:
1. Enable a different update channel for aries dogfood
2. Only allow balrog for aries ota on mozilla-central and mozilla-b2g44_v2.5 (i.e. don't allow it on b2g-ota)
3. allow additional-parameters to be specified in job_flags
4. remove balrogvpn scopes and features from tasks that do not need them.  The fota and ota config files for flame-kk/aries have the balrog step commented out so these should not be needed anymore
Attached file b2g-ota graph
Here is the graph that's generated for b2g-ota
Attachment #8693936 - Flags: feedback?(catlee)
Attachment #8693938 - Flags: feedback?(mpotharaju)
Attachment #8693938 - Flags: feedback?(catlee)
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

https://reviewboard.mozilla.org/r/26623/#review24145

lgtm
Attachment #8693936 - Flags: review?(wcosta) → review+
(In reply to Mahendra Potharaju [:mahe] from comment #11)
>  (In reply to Greg Arndt [:garndt] from comment #10)
> > It appears that based on commend 2, Aries and Flame are what we're concerned
> > with in this branch. 
> 
> Please let me clarify my comment 2 here. 
> We want to BEGIN with Aries and Flame, but be able to provide updates to
> Nexus 4,5,6 and other phones in the future once we have the How, When and
> other related questions answered.

I think we should open separate bugs for these so that we don't have bugs open forever that are difficult to track.
Ok, so after talking with catlee and wcosta today, it seems that there are cases where changes are checked into the branch specifically and not through a flow of inbound -> m-c -> release branch.

These changes were under the assumption that changes from m-c gets merged into that branch and there wasn't special cases on that branch.  As it has been discovered, there are configs only on certain branches, such as b2g44 that would get broken by this patch because of the update_channel being ""nightly-b2g44"" only on that branch and not in configs elsewhere.

So from my understanding for the ***Aries dogfood*** build, the channels should be as such for the following branches:
mozilla-central
b2g_update_channel = dogfood

mozilla-b2g44_v2_5
b2g_update_channel = nightly_b2g44

b2g-ota
b2g_update_channel = b2g-ota
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/26623/diff/1-2/
Attachment #8693936 - Flags: feedback?(catlee)
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

So I hope that this will get the right channels, please double, triple check some of this as I think you'll need to run with this patch this week while I'm out.

Right now b2g44 is a special snowflake in that it has branch specific configs in mozharness for the channel to use.  I mainly added the b2g44 update channel stuff in this just in case someone tries to merge these changes into that branch.  

From what I understand m-c should be under nightly, and other branches should be more branch specific as to the channel to use.
Attachment #8693936 - Flags: review+ → review?(wcosta)
Attachment #8693936 - Flags: review?(wcosta) → review+
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

https://reviewboard.mozilla.org/r/26623/#review24173

Ok, it looks safe. Could you please request catlee's feedback again?
This change has landed on b2g-i and scheduling is working for b2g-ota.  Once green on b2g-i wander is going to land on b2g-ota.
https://hg.mozilla.org/mozilla-central/rev/1003fca97839
https://hg.mozilla.org/releases/b2g-ota/rev/26ae868bf536

Hope this works by just manually cherry-picking the patch around.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla45
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/26623/diff/2-3/
Attachment #8693936 - Attachment description: MozReview Request: Bug 1227355 - Configure b2g-ota branch and add update channel r=wcosta → MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee
Attachment #8693936 - Flags: review?(catlee)
Attachment #8693938 - Flags: feedback?(mpotharaju)
Attachment #8693938 - Flags: feedback?(catlee)
Comment on attachment 8693936 [details]
MozReview Request: Bug 1227355 - Remove update channel environment variable from aries device config r=catlee

https://reviewboard.mozilla.org/r/26623/#review24213
Attachment #8693936 - Flags: review?(catlee) → review+
https://hg.mozilla.org/integration/b2g-inbound/rev/a07184a2fadb9bc60929757dc1113dc262a4758a
Bug 1227355 - Remove update channel environment variable from aries device config r=catlee
I'm on PTO for the rest of the week but wcosta is around and can help if something is missed. I double checked the builds and it looks like the right update channel is now being used for aries (flake-kk is a whole different issue).  Let us know if there is anything more we can do to help get things right.
Component: Service Request → Operations and Service Requests
You need to log in before you can comment on or make changes to this bug.