1264006 - release promotion - create part 2 of RC firefox task graph

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Comment 1

[Jordan Lund (:jlund)]

Attachment: MozReview Request: Bug 1264006 - release promotion - tools - create part 2 of RC firefox task graph in release tasks, r?rail

Review commit: https://reviewboard.mozilla.org/r/45813/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/45813/

Comment 2

[Jordan Lund (:jlund)]

Attachment: MozReview Request: Bug 1264006 - release promotion - bbot-cfgs create part 2 of RC firefox task graph

we will need to update this when we productionify mozilla-release configs

Review commit: https://reviewboard.mozilla.org/r/45821/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/45821/

Comment 3

[Jordan Lund (:jlund)]

Attachment: Bug 1264006 - release promotion - releasetasks create part 2 of RC firefox task graph

Comment 4

[Jordan Lund (:jlund)]

Comment on attachment 8740523 [details]
MozReview Request: Bug 1264006 - release promotion - tools - create part 2 of RC firefox task graph in release tasks, r?rail

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/45813/diff/1-2/

Comment 5

[Rail Aliiev [:rail]]

Comment on attachment 8740524 [details]
MozReview Request: Bug 1264006 - release promotion - bbot-cfgs create part 2 of RC firefox task graph

https://reviewboard.mozilla.org/r/45821/#review42503

::: mozilla/config.py:2625
(Diff revision 1)
>  BRANCHES['mozilla-beta']['platforms']['win32']['dep_signing_servers'] = 'release-signing'
>  BRANCHES['mozilla-beta']['platforms']['win64']['dep_signing_servers'] = 'release-signing'
>  # used by releasetasks
>  BRANCHES['mozilla-beta']['bouncer_enabled'] = True
> +BRANCHES['mozilla-beta']['updates_builder_enabled'] = True
> +BRANCHES['mozilla-beta']['update_verify_enabled'] = True

I think I need to port these to bug 1263922

Comment 6

[Rail Aliiev [:rail]]

Comment on attachment 8740523 [details]
MozReview Request: Bug 1264006 - release promotion - tools - create part 2 of RC firefox task graph in release tasks, r?rail

https://reviewboard.mozilla.org/r/45813/#review42505

It'd be great to apply the final patches to dev release runner and start a release.

::: buildfarm/release/kickoff_configs/dev_mozilla-release_firefox_rc_graph_2.py:27
(Diff revision 2)
> +    "postrelease_version_bump_enabled": True,
> +
> +    "product": "firefox",
> +    "repo_path": "projects/jamun",
> +    "funsize_balrog_api_root": "http://ec2-54-241-39-23.us-west-1.compute.amazonaws.com:443/api",
> +    "balrog_api_root": "http://ec2-54-241-39-23.us-west-1.compute.amazonaws.com:443/api",

it looks werid to have these configs here...
Maybe instead you read them from branch config as in release-runner and override some of them in this file?

::: buildfarm/release/releasetasks_graph_gen.py:49
(Diff revision 2)
> +    docker_worker_key = get_config(config, 'release-runner',
> +                                   'docker_worker_key', None)
> +    signing_pvt_key = get_config(config, 'signing', 'pvt_key', None)
> +    if isinstance(notify_to, basestring):
> +        notify_to = [x.strip() for x in notify_to.split(',')]
> +    smtp_server = get_config(config, 'release-runner', 'smtp_server',

I think this and `email_release_drivers` can go away from this file.

::: buildfarm/release/releasetasks_graph_gen.py:142
(Diff revision 2)
> +        print scheduler.createTaskGraph(graph_id, graph)
> +
> +        email_release_drivers(smtp_server=smtp_server, from_=notify_from,
> +                              to=notify_to, release=release_config,
> +                              graph_id=graph_id)
> +    except:

I think you can remove the try/except wrapper and let it fail.

::: buildfarm/release/releasetasks_graph_gen.py:166
(Diff revision 2)
> +    parser = OptionParser(__doc__)
> +    parser.add_option('-c', '--config', dest='config',
> +                      help='Configuration file')
> +    parser.add_option('--release-config', dest='release_config',
> +                      help='Release Specific Configuration file')
> +    parser.add_option('--version', dest='version',

That's a lot of things to pass. I wonder if we can pass graph 1 ID instead and use it to fetch this info? Any of graph 1 tasks contains enough information in `task.extra.build_props`.

::: buildfarm/release/releasetasks_graph_gen.py:190
(Diff revision 2)
> +
> +    release_config = get_release_config(options.release_config)
> +
> +    l10n_changesets = {}
> +    if options.l10n_changesets_file:
> +        log.fatal("--l10n-changesets-file is not supported yet")

Sounds like you either want to use `log.critical()` or `parser.error()` instead of `log.fatal()` here.

::: buildfarm/release/releasetasks_graph_gen.py:191
(Diff revision 2)
> +    release_config = get_release_config(options.release_config)
> +
> +    l10n_changesets = {}
> +    if options.l10n_changesets_file:
> +        log.fatal("--l10n-changesets-file is not supported yet")
> +        # TODO get contents from file and pass to parsePlainL10nChangesets

maybe use the ship-it API to fetch the l10n changesets? Just like in `get_l10n_config()`

::: lib/python/kickoff/__init__.py:133
(Diff revision 2)
> +def get_partials(rr, partial_versions, product):
> +    partials = {}
> +    if not partial_versions:
> +        return partials
> +    for p in partial_versions.split(','):
> +        partialVersion, buildNumber = p.split('build')

This has been always bothering me. While you are on this, can you strip() before splitting:

`partialVersion, buildNumber = p.strip().split('build')`

so we can handle coma + space separated partials

::: lib/python/kickoff/__init__.py:167
(Diff revision 2)
> +            platform=platform,
> +        ))
> +        url = "https://queue.taskcluster.net/v1/task/{taskid}/artifacts/public/build".format(
> +            taskid=task["taskId"]
> +        )
> +        l10n_platforms[platform] = {

Typo? Should be `l10n_platform_configs`

Comment 7

[Jordan Lund (:jlund)]

Comment on attachment 8740523 [details]
MozReview Request: Bug 1264006 - release promotion - tools - create part 2 of RC firefox task graph in release tasks, r?rail

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/45813/diff/2-3/

Comment 8

[Jordan Lund (:jlund)]

Comment on attachment 8740524 [details]
MozReview Request: Bug 1264006 - release promotion - bbot-cfgs create part 2 of RC firefox task graph

thanks, https://hg.mozilla.org/build/buildbot-configs/rev/27320b5ae218

Comment 9

[Jordan Lund (:jlund)]

Comment on attachment 8740523 [details]
MozReview Request: Bug 1264006 - release promotion - tools - create part 2 of RC firefox task graph in release tasks, r?rail

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/45813/diff/3-4/

Comment 10

[Jordan Lund (:jlund)]

Comment on attachment 8740525 [details] [review]
Bug 1264006 - release promotion - releasetasks create part 2 of RC firefox task graph

new stuff :)

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 8740523 [details]
MozReview Request: Bug 1264006 - release promotion - tools - create part 2 of RC firefox task graph in release tasks, r?rail

https://reviewboard.mozilla.org/r/45813/#review43695

Comment 12

[Jordan Lund (:jlund)]

we need to create a production version of https://github.com/mozilla/releasetasks/blob/master/releasetasks/release_configs/dev_mozilla-release_firefox_rc_graph_2.yml

will do that today

Comment 13

[Rail Aliiev [:rail]]

anything left here?