Closed Bug 1277595 Opened 8 years ago Closed 7 years ago

Create task graph for Fennec nightlies

Categories

(Firefox Build System :: Task Configuration, task)

Product:
Firefox Build System
Component:
Task Configuration
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla51

People

(Reporter: coop, Assigned: amiyaguchi)

References

Details

Attachments

(6 files, 12 obsolete files)

Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph,
58 bytes, text/x-review-board-request
dustin
: review+
Details
Bug 1277595 - Enable multil10n fennec builds on taskcluster,
58 bytes, text/x-review-board-request
jlund
: review+
Details
Bug 1277595 - Use mozilla-central config-file on try,
58 bytes, text/x-review-board-request
jlund
: review+
Details
Bug 1277595 - Generate balrog properties as a taskcluster artifact,
58 bytes, text/x-review-board-request
jlund
: review+
Details
Bug 1277595 - Create initial nightly graph,
58 bytes, text/x-review-board-request
jlund
: review+
Details
Bug 1277595 - Update signing task to use signing-linux-v1 worker,
58 bytes, text/x-review-board-request
jlund
: review+
Details 
We're getting close on Fennec builds, so we should start thinking about the nightly story here.
Blocks: 1277682
No longer depends on: 1118394
Depends on: 1282180
Depends on: 1282186
Depends on: 1282192
Depends on: 1282193
Depends on: 1282197
Depends on: 1282200
Summary: Create task graph for Fennec nightlies → Create task graph for Linux nightlies (includes Fennec)
Splitting these back out again. Sounds like we can re-use several kinds, but not everything.
Summary: Create task graph for Linux nightlies (includes Fennec) → Create task graph for Fennec nightlies
Anthony, Jordan: do we need *all* of the tasks in place just to have a working graph, or can we stub stuff out and fill it in as we go?

Or phrased another way, should all of the various sub-task bugs be blocking a "Fennec nightly as Tier 2" bug instead of this bug?
Flags: needinfo?(jlund)
Flags: needinfo?(amiyaguchi)
(In reply to Chris Cooper [:coop] from comment #3)
> Anthony, Jordan: do we need *all* of the tasks in place just to have a
> working graph, or can we stub stuff out and fill it in as we go?
> 

we put all of these as blocking to serve as a reminder what was needed in the graph. now that we are tracking this in trello and we have a draw.io visualization, I'll clean up the dep tree so that this bug can represent the minimum impl required.
Flags: needinfo?(jlund)
The graph won't be very useful until the tasks within it are complete. I have some work that is relevant to this (creating a nightly fennec kind with dependencies) with stubs for signing and balrog, but creating the graph can't be finalized until the various components are working.
Flags: needinfo?(amiyaguchi)
This adds a triggered-by option to the decision command which allows for the
use of a nightly method for specifying the target task set. The nightly method
is currently a stub.

Review commit: https://reviewboard.mozilla.org/r/64648/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/64648/
This is meant to be a stand in for the build kind while the in-tree
configuration tool goes under a large refactoring.

Review commit: https://reviewboard.mozilla.org/r/64650/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/64650/
For some reason, the decision task doesn't seem to be able to assign tasks to
the workerType of `tutorial`. Timestamps have also been changed to reflect the
behavior of othere similar templates.

Review commit: https://reviewboard.mozilla.org/r/64652/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/64652/
Disable upload symbols in the build system if the buildpool is taskcluster. The
build system wants to upload symbols to soccorro, but is missing the proper
token for doing so. There is a separate task to do this.

Review commit: https://reviewboard.mozilla.org/r/64658/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/64658/
Assignee: nobody → amiyaguchi
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64658/diff/1-2/
No longer depends on: 1282186, 1282192, 1282193, 1282197, 1282200
Depends on: 1286989
No longer depends on: 1286989
https://reviewboard.mozilla.org/r/64648/#review62332

::: taskcluster/taskgraph/decision.py:112
(Diff revision 1)
>                         "PER_PROJECT_PARAMETERS in {} to customize behavior "
>                         "for this project".format(project, __file__))
>          parameters.update(PER_PROJECT_PARAMETERS['default'])
>  
> +    if parameters['triggered_by'] == 'nightly':
> +        parameters['target_tasks_method'] = 'nightly'

I'd rather see this option passed explicitly along with --triggered-by

::: taskcluster/taskgraph/target_tasks.py:52
(Diff revision 1)
>      return [t.label for t in full_task_graph.tasks.itervalues()
>              if t.attributes.get('kind') == 'legacy']
> +
> +@_target_task('nightly')
> +def target_tasks_nightly(full_task_graph, parameters):
> +    """Generate the target set for the creation of the nightly graph."""

Probably something like "select the set of tasks required for a nightly build: .." with some description of which tasks those are.
https://reviewboard.mozilla.org/r/64650/#review62336

I can't quite make out what the nightly-fennec kind is -- is it a variety of build that passes parameters to mozharness to trigger nightly branding, etc.?  I worry that might lead to duplication with the build kind, with things getting out of sync.

::: taskcluster/ci/balrog/kind.yml:9
(Diff revision 1)
> +
> +implementation: 'taskgraph.kind.balrog:BalrogTask'
> +balrog_path: '.'
> +kind-dependencies:
> +  - nightly-fennec
> +  - signing

Remember that these kind dependencies don't affect task dependencies.  It's only useful if the kind implementation wants to look at tasks already generated by another kind.  Signing might make sense here if you want to generate a balrog task for every signing task.
https://reviewboard.mozilla.org/r/64658/#review62340

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:866
(Diff revision 2)
>          env['MOZ_SOURCE_REPO'] = repo_path
>  
> +        # If we are building on taskcluster, do not upload symbols via the
> +        # build process.
> +        if self.config.get('taskcluster'):
> +            env["MOZ_AUTOMATION_UPLOAD_SYMBOLS"] = 0

Did bug 1278445 fix the same problem?
Sorry for the preemptive comments -- I'm curious what you're up to :)

Also, with the landing of bug 1281004, you'll want to rebase and, I expect, use the transform-based task-generation mechanisms rather than the tempated-yaml-based methods from legacy.  For kinds as simple as docker-image, I can be convinced to use a YAML template (just one -- no inheritance).  It may be that signing and balrog are this simple, with every task looking almost identical but with some different parameters.  Even there, it might make more sense to use a YAML template that creates a task description (see the docs in bug 1281004 for what "task description" means) and let the transforms take it from there.
No longer depends on: 1282180
Depends on: 1282186, 1282187, 1282192
Thank you for the review Dustin, I appreciate it :).

For more context, the nightly-fennec kind is temporary until the build kind is ready to go. I am currently working on finding the right configs and flags to get nightly fennec building on TaskCluster. I did end up finding bug 1278445, which solves the problems I had with symbol uploading. I will swap out this kind for the refactored build kind once it reaches completion.

The only difference between a nightly and a CI build seems to only be a few environmental variables, especially since we're factoring out the nightly specific mozharness actions into their own task. I was thinking it might be neat if I could define a "nightly" transformation that took the CI task definition and swapped the mozharness parameters for the ones necessary for nightly. It would be useful to have this tagged as it's own kind so my signing kind can generated its tasks for these nightly builds. 

The signing and balrog kinds are hopefully straightforward. It will probably be a matter of having tasks generate their own signing manifest (a blob containing files to sign and their hashes), and having the scriptworker look for the artifacts of its dependency.
I think that transformation is a good idea, however I do not think it should be a distinct kind.  Rather, the build kind should have that transformation, but only apply it when params['triggered_by'] == 'nightly'.
Comment on attachment 8771520 [details]
Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64648/diff/1-2/
Attachment #8771521 - Attachment description: Bug 1277595 - Add a temporary nightly-fennec kind → Bug 1277595 - Create initial nightly graph
Attachment #8771522 - Attachment description: Bug 1277595 - Change workerType and timestamps to run correctly → Bug 1277595 - Enable multil10n fennec builds on taskcluster
Attachment #8771524 - Attachment description: Bug 1277595 - Enable nightly variable in mozharness builds → Bug 1277595 - Do not upload symbols for tasks on taskcluster
Attachment #8771525 - Attachment description: Bug 1277595 - Disable upload symbols for nightly mozharness builds → Bug 1277595 - Use mozilla-central config-file on try
Comment on attachment 8771523 [details]
Bug 1277595 - Update signing task to use signing-linux-v1 worker

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64654/diff/1-2/
Comment on attachment 8771521 [details]
Bug 1277595 - Create initial nightly graph

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64650/diff/1-2/
Comment on attachment 8771522 [details]
Bug 1277595 - Enable multil10n fennec builds on taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64652/diff/1-2/
Comment on attachment 8771524 [details]
Bug 1277595 - Do not upload symbols for tasks on taskcluster

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64656/diff/1-2/
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/64658/diff/2-3/
https://reviewboard.mozilla.org/r/64652/#review64846

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:1663
(Diff revision 2)
> +        if self.config.get('taskcluster_nightly'):
> +            multi10n_path = \
> +                'build/src/testing/mozharness/scripts/multil10n.py'
> +            base_work_dir = os.path.join(base_work_dir, 'workspace')
> +        else:
> +            multi10n_path = '%s/scripts/scripts/multil10n.py' % base_work_dir,

You can (probably should) do something like:

https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/mediatests/buildbot_posix_config.py#5

for where the multil10n.py file is, rather than rely on the taskcluster_nightly flag and specifying the whole path differently.
These series of patches now require the use of a hook with the proper roles that can access script-worker. The role hook-id:releng/nightly-fennec-dev defines the required scopes listed below.

>    assume:repo:hg.mozilla.org/try:*
>    project:releng:signing:cert:dep-signing
>    project:releng:signing:format:gpg
>    queue:create-task:scriptworker-prov-v1/dummy-worker-miya1
Depends on: 1282180
Signing for en-US and multi10n are working at https://tools.taskcluster.net/task-inspector/#CU6ZNCBTTAiS6Vgk-TFFvQ/0. Thanks Aki :)

I've current been working on integrating the funsize-balrog worker into the nightly graph. I currently have the balrogworker deployed and consuming tasks for `dummy-task-provisioner/dummy-worker-balrog`. The task defined at https://tools.taskcluster.net/task-inspector/#IxpynluMS2a8BJoUYBzMoA/ lets me know that the worker has been set up correctly.

I've written up how to set up your own balrogworker and balrog instances. If you happen to read through it, I'd gladly take feedback. https://git.io/v61F7

The current problem I'm tackling is that the balrogworker expects a manifest that should be generated in the build step. However, if the artifacts are signed along the way then the manifest will be out of date when it reaches the balrogworker. The signing task should take both the build artifacts and the manifest, and update the meta-information. This way the output from the build step and the signing step will be consistent and equivalent from the balrogworker's point of view.

I will need to generate a valid manifest after creating a nightly build in buildbase.py. The manifest should be updated with the location of the signed artifacts. For now, I will avoid moving artifacts to s3 and also whitelist TaskCluster urls in balrog. My local balrog instance should then be aware of the nightly locations in taskcluster if everything is set up correctly.
Attachment #8771524 - Attachment is obsolete: true
Comment on attachment 8771522 [details]
Bug 1277595 - Enable multil10n fennec builds on taskcluster

https://reviewboard.mozilla.org/r/64652/#review71776

looks good. just one typo. ship eet :)

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:1659
(Diff revision 3)
> +            multi10n_path = \
> +                'build/src/testing/mozharness/scripts/multil10n.py'
> +            base_work_dir = os.path.join(base_work_dir, 'workspace')
> +        else:
> +            multi10n_path = '%s/scripts/scripts/multil10n.py' % base_work_dir,
>  
>          cmd = [
>              self.query_exe('python'),
> -            '%s/scripts/scripts/multil10n.py' % base_work_dir,
> +            multi10n_path,

s/multi10n_path/multil10n_path/
Attachment #8771522 - Flags: review?(jlund) → review+
I've rebased these patches and verified that the graph can be generated from a decision task. The task and subsequent taskgraph can be found at https://tools.taskcluster.net/task-inspector/#ZF39-Hj8RIaLIato6NZGvA.

This will generate the nightly task graph with a temporary kind for nightly android-api-15 builds. Apk signing with signing scriptworker is functional. Generating a balrog manifest for funsize-balrogworker is still in progress; the manifest generated in the build task and in buildbot are different.

Once legacy build kinds are properly depreciated, the temporary nightly kind should go away. With triggered-by parameter from the decision task, we should be able to conditionally transform the android-api-15 task into a nightly task and use it as the basis for the task graph.
Comment on attachment 8771523 [details]
Bug 1277595 - Update signing task to use signing-linux-v1 worker

https://reviewboard.mozilla.org/r/64654/#review72128

r+ with nits, largely around making signing.yml reusable.

::: taskcluster/ci/signing/signing.yml:6
(Diff revision 4)
>  task:
> -  provisionerId: "aws-provisioner-v1"
> -  workerType: "desktop-test"
> -  schedulerId: task-graph-scheduler
> -
> +  provisionerId: "scriptworker-prov-v1"
> +  workerType: "signing-linux-v1"
> +  scopes:
> +    - "project:releng:signing:cert:dep-signing"
> +    - "project:releng:signing:format:jar"

If we share this yaml file, then we'll have to be more flexible with scopes.

In CI, we want `project:releng:signing:cert:dep-signing`, but for nightlies we'll want `project:releng:signing:cert:nightly-signing` and releases will have `project:releng:signing:cert:release-signing`.

For Android APKs, we want `project:releng:signing:format:jar`, but for linux signing we'll want `project:releng:signing:format:gpg`, and we'll want different signing formats for dmgs, exes, addons, etc.

We could land with the file as-is, but we would need a followup bug to track this work so the next graph (linux 64?) can take advantage.

::: taskcluster/ci/signing/signing.yml:16
(Diff revision 4)
> -      - "-c"
> -      - "echo \"hello World\""
>      maxRunTime: 600
>    metadata:
> -    name: "Signing Fennec Nightly Task"
> -    description: "Markdown description of **what** this task does"
> +    name: "Signing Scriptworker Task"
> +    description: "Testing the signing scriptworker"

The name could possibly stay `Signing Fennec Nightly Task` so we can differentiate it from other Signing Scriptworker Tasks.  However, if we share it with other graphs, then we either templatize it, update it in the actual graph generation, or maybe we live with the generic name.

The description is accurate for now, but once we roll out we may want a more apt description.
Attachment #8771523 - Flags: review?(aki) → review+
Comment on attachment 8771520 [details]
Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph

https://reviewboard.mozilla.org/r/64648/#review72394

Minor detail to fix..

::: taskcluster/taskgraph/decision.py:126
(Diff revision 4)
>          logger.warning("using default project parameters; add {} to "
>                         "PER_PROJECT_PARAMETERS in {} to customize behavior "
>                         "for this project".format(project, __file__))
>          parameters.update(PER_PROJECT_PARAMETERS['default'])
>  
> +    # `target_tasks_method` has higher precedence than `project` parameters

If so, then it shouldn't have a default value, as that will always override the per project parameters :)
Attachment #8771520 - Flags: review?(dustin) → review-
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

https://reviewboard.mozilla.org/r/64658/#review72454

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:1651
(Diff revision 6)
>          branch = self.branch
>  
> +        # Building a nightly with the try repository fails because a
> +        # config-file does not exist for try. Default to mozilla-central
> +        # settings (arbitrarily).
> +        if branch == 'try':

this hack won't work once we want to do aurora nightlies on try.. :(

is there another way we can determine that the try build is based off central or aurora?
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

https://reviewboard.mozilla.org/r/64658/#review72454

> this hack won't work once we want to do aurora nightlies on try.. :(
> 
> is there another way we can determine that the try build is based off central or aurora?

I think we could probably use the taskcluster environment variable `GECKO_HEAD_REPOSITORY` to determine the branch. That way the information about the respository we want to use and the branch it's actually running in are independent.
Comment on attachment 8771521 [details]
Bug 1277595 - Create initial nightly graph

https://reviewboard.mozilla.org/r/64650/#review72490

of rough view, this looks good. There are parts of the signing and balrog definitions that have hardcoded 'fennec' and 'nightly' references. Maybe they should be more generic?

I don't think I should be the final person to sign off on this. I'd love Dustin's stamp. Could you add him to the review list against this cset?
Attachment #8771521 - Flags: review?(jlund) → review+
Comment on attachment 8784953 [details]
Bug 1277595 - Generate balrog properties as a taskcluster artifact

https://reviewboard.mozilla.org/r/74272/#review72494

nice. I like this patch!
Attachment #8784953 - Flags: review?(jlund) → review+
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

https://reviewboard.mozilla.org/r/64658/#review72454

> I think we could probably use the taskcluster environment variable `GECKO_HEAD_REPOSITORY` to determine the branch. That way the information about the respository we want to use and the branch it's actually running in are independent.

yep. let's go for that. and can we add a log line explaining the side-effect mutation of branch. Also, can we wrap the whole thing around a `if taskcluster_nightly:` block. Not needed but at least it is more explicit while we juggle both bbot and tc against the same script.
Comment on attachment 8771521 [details]
Bug 1277595 - Create initial nightly graph

dustin: anthony just finished his internship. I'm going to be making sure this lands by fixing up patches in the review process. Could you have a look at this as per my suggestion in https://bugzilla.mozilla.org/show_bug.cgi?id=1277595#c53

not sure how I can clear this up or take over in mozreview but I can at least start by setting the correct flags in bz.
Attachment #8771521 - Flags: review+ → review?(dustin)
Comment on attachment 8771520 [details]
Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph

https://reviewboard.mozilla.org/r/64648/#review73164
Attachment #8771520 - Flags: review?(dustin) → review+
Comment on attachment 8771521 [details]
Bug 1277595 - Create initial nightly graph

I'm not sure why I'm still flagged for review here.  I'm hesitant to land the temporary nightly kind, without knowing what's next for this.  Maybe we should talk about this out of bugzilla, jordan?
oh fun, I guess I can't update other OP mozreviews.

comment 60-65, https://reviewboard.mozilla.org/r/75796/, is a graft of Anthony's original csets in https://reviewboard.mozilla.org/r/64646/

I'll carry forward the already reviewed bits
Comment on attachment 8771522 [details]
Bug 1277595 - Enable multil10n fennec builds on taskcluster

https://reviewboard.mozilla.org/r/64652/#review64846

> You can (probably should) do something like:
> 
> https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/mediatests/buildbot_posix_config.py#5
> 
> for where the multil10n.py file is, rather than rely on the taskcluster_nightly flag and specifying the whole path differently.

is this still an issue callek? iiuc because we are calling a mh script within mh, we reference off of base_work_dir to reach the script. But in TC world, we want to (1) run this one level deeper in base_work_dir: `workspace` and (2) use a different mh checkout path. `os.path.abspath(os.path.dirname(os.path.dirname(mozharness.__file__)))` is sort of a hack that's required outside of the script because we don't have query_abs_dirs attr to leverage.

Maybe we should have these paths defined at the config level? Could we follow up in a further bug for this?
ni? comment 67
Flags: needinfo?(bugspam.Callek)
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

https://reviewboard.mozilla.org/r/64658/#review73768
Attachment #8771525 - Flags: review?(jlund) → review+
Comment on attachment 8771525 [details]
Bug 1277595 - Use mozilla-central config-file on try

https://reviewboard.mozilla.org/r/64658/#review72454

> yep. let's go for that. and can we add a log line explaining the side-effect mutation of branch. Also, can we wrap the whole thing around a `if taskcluster_nightly:` block. Not needed but at least it is more explicit while we juggle both bbot and tc against the same script.

I'll file a follow up bug for this
Comment on attachment 8786957 [details]
Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph,

https://reviewboard.mozilla.org/r/75798/#review73760

carrying r+ from dustin here: https://reviewboard.mozilla.org/r/64648/diff/5#index_header
Attachment #8786957 - Flags: review+
Comment on attachment 8786958 [details]
Bug 1277595 - Enable multil10n fennec builds on taskcluster

https://reviewboard.mozilla.org/r/75800/#review73770

carrying forward r+ from jlund here: https://reviewboard.mozilla.org/r/64652/diff/5#index_header
Attachment #8786958 - Flags: review+
Comment on attachment 8786959 [details]
Bug 1277595 - Use mozilla-central config-file on try

https://reviewboard.mozilla.org/r/75802/#review73772

carrying forward jlund's r+ from here: https://reviewboard.mozilla.org/r/64658/diff/6#index_header
Attachment #8786959 - Flags: review+
Comment on attachment 8786960 [details]
Bug 1277595 - Generate balrog properties as a taskcluster artifact

https://reviewboard.mozilla.org/r/75804/#review73774

carrying forward jlund's r+ from here: https://reviewboard.mozilla.org/r/74272/diff/2#index_header
Attachment #8786960 - Flags: review+
Comment on attachment 8786962 [details]
Bug 1277595 - Update signing task to use signing-linux-v1 worker

https://reviewboard.mozilla.org/r/75808/#review73776

carrying forward aki's r+ from here: https://reviewboard.mozilla.org/r/64654/diff/5#index_header
Attachment #8786962 - Flags: review+
Attachment #8786961 - Flags: review?(dustin)
Comment on attachment 8786961 [details]
Bug 1277595 - Create initial nightly graph

https://reviewboard.mozilla.org/r/75806/#review73786

sigh, I've lost micro diffs since editing Anthony's original in https://reviewboard.mozilla.org/r/64650

dustin: basically as we discussed over vidyo, this keeps the temp fennec-nightly kind but removes the incomplete balrog stub kind. For now :)
Attachment #8786961 - Flags: review?(jlund)
Comment on attachment 8786961 [details]
Bug 1277595 - Create initial nightly graph

grr
Attachment #8786961 - Flags: review?(jlund)
Attachment #8771520 - Attachment is obsolete: true
Attachment #8771521 - Attachment is obsolete: true
Attachment #8771521 - Flags: review?(dustin)
Attachment #8771522 - Attachment is obsolete: true
Attachment #8771523 - Attachment is obsolete: true
Attachment #8771525 - Attachment is obsolete: true
Attachment #8784953 - Attachment is obsolete: true
Comment on attachment 8771522 [details]
Bug 1277595 - Enable multil10n fennec builds on taskcluster

https://reviewboard.mozilla.org/r/64652/#review64846

> is this still an issue callek? iiuc because we are calling a mh script within mh, we reference off of base_work_dir to reach the script. But in TC world, we want to (1) run this one level deeper in base_work_dir: `workspace` and (2) use a different mh checkout path. `os.path.abspath(os.path.dirname(os.path.dirname(mozharness.__file__)))` is sort of a hack that's required outside of the script because we don't have query_abs_dirs attr to leverage.
> 
> Maybe we should have these paths defined at the config level? Could we follow up in a further bug for this?

I would love if we fixed this now, but given how much of a mess the l10n code is, I don't mind a followup bug if doing so helps us unblock the current nightly stuff. (There is already a bunch of cleanup around l10n to do, and this wouldn't tip scales significantly)
Comment on attachment 8786961 [details]
Bug 1277595 - Create initial nightly graph

https://reviewboard.mozilla.org/r/75806/#review73968

Looks good, thanks!
Attachment #8786961 - Flags: review?(dustin) → review+
Flags: needinfo?(bugspam.Callek)
https://hg.mozilla.org/integration/mozilla-inbound/rev/e7dd86addfd5606de5e6236f4241e8942c14d048
backed out Bug 1277595, revs d5d26aa0f369, 42ab74605817, c1aa2a15b4eb, 32a569f4df06, dee2532bdc95, 5809a56922ed CLOSED TREE
Attachment #8786958 - Attachment is obsolete: true
Attachment #8786959 - Attachment is obsolete: true
Attachment #8786960 - Attachment is obsolete: true
Attachment #8786961 - Attachment is obsolete: true
Attachment #8786962 - Attachment is obsolete: true
Comment on attachment 8787366 [details]
Bug 1277595 - Enable multil10n fennec builds on taskcluster,

https://reviewboard.mozilla.org/r/76144/#review74208
Attachment #8787366 - Flags: review?(jlund) → review+
Comment on attachment 8787367 [details]
Bug 1277595 - Use mozilla-central config-file on try,

https://reviewboard.mozilla.org/r/76146/#review74210
Attachment #8787367 - Flags: review?(jlund) → review+
Comment on attachment 8787367 [details]
Bug 1277595 - Use mozilla-central config-file on try,

https://reviewboard.mozilla.org/r/76146/#review74212
Comment on attachment 8787368 [details]
Bug 1277595 - Generate balrog properties as a taskcluster artifact,

https://reviewboard.mozilla.org/r/76148/#review74214
Attachment #8787368 - Flags: review?(jlund) → review+
Comment on attachment 8787369 [details]
Bug 1277595 - Create initial nightly graph,

https://reviewboard.mozilla.org/r/76150/#review74216
Attachment #8787369 - Flags: review+
Comment on attachment 8787370 [details]
Bug 1277595 - Update signing task to use signing-linux-v1 worker,

https://reviewboard.mozilla.org/r/76152/#review74220
Attachment #8787370 - Flags: review+
Comment on attachment 8787369 [details]
Bug 1277595 - Create initial nightly graph,

sorry for the noise
Attachment #8787369 - Flags: review?(dustin)
Comment on attachment 8787370 [details]
Bug 1277595 - Update signing task to use signing-linux-v1 worker,

sorry for the noise
Attachment #8787370 - Flags: review?(aki)
Comment on attachment 8786957 [details]
Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph,

okay ... I officially give up.

dustin: comment 82-95 is simply to get a r? stamp for: https://reviewboard.mozilla.org/r/75798/diff/1-2/
Comment on attachment 8786957 [details]
Bug 1277595 - Add a --triggered-by=nightly flag to taskgraph,

https://reviewboard.mozilla.org/r/75798/#review74396

Ah, I see -- options will always have all of its keys, just maybe with the value None if they were not specified.
Attachment #8786957 - Flags: review?(dustin) → review+
Depends on: 1300168
okay, so this bug is a little bloated. The patches in comment 98 represent the goal: creating an init task graph for nightlies. Will continue on filling in the graph in subsequent bugs.

I also filed 'Bug 1299859 - fennec nightlies should reference multil10n.py path at config level' for:

(In reply to Justin Wood (:Callek) from comment #78)
> Comment on attachment 8771522 [details]
> Bug 1277595 - Enable multil10n fennec builds on taskcluster
> 
> https://reviewboard.mozilla.org/r/64652/#review64846
> 
> > is this still an issue callek? 
> I would love if we fixed this now, but given how much of a mess the l10n
> code is, I don't mind a followup bug if doing so helps us unblock the
> current nightly stuff.


and 'Bug 1299856 - tc fennec nightly try runs should use GECKO_HEAD_REPOSITORY to determine the branch for multil10n' for:

(In reply to Jordan Lund (:jlund) from comment #55)
> Comment on attachment 8771525 [details]
> Bug 1277595 - Use mozilla-central config-file on try
> 
> https://reviewboard.mozilla.org/r/64658/#review72454
> 
> > I think we could probably use the taskcluster environment variable `GECKO_HEAD_REPOSITORY` to determine the branch. That way the information about the respository we want to use and the branch it's actually running in are independent.

and finally 'Bug 1300168 - clean up fennec nightly and signing flake errors' :)
See Also: → 1299859, 1299856
Depends on: 1300795
Blocks: 1302590
Blocks: 1321005
Depends on: 1336864
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.