1378252 - Enable funsize for TC osx builds/repacks

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement)

Description

[Rail Aliiev [:rail]]

Mac builds are generated by TC and there are some differences versus TC linux builds/repacks:

1) complete.mar files are generated in repackage-signing tasks (vs build-signing on linux). The tasks should be "marked" as funsize tasks.

2) repackage-signing has more than one dependency. The funsize scheduler should find the "build" task, which contains gecko revision info and balrog_props.json

Comment 1

[Rail Aliiev [:rail]]

Attachment: Bug 1378252 - Add funsize routes to mac repackage-signing tasks

Review commit: https://reviewboard.mozilla.org/r/154326/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/154326/

Comment 2

[Rail Aliiev [:rail]]

Attachment: Funsize scheduler PR

Comment 3

[Rail Aliiev [:rail]]

Attachment: Bug 1378252 - Bump funsize version

Review commit: https://reviewboard.mozilla.org/r/154510/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/154510/

Comment 4

[Simon Fraser [:sfraser] ⌚️GMT]

Comment on attachment 8883571 [details]
Bug 1378252 - Bump funsize version

https://reviewboard.mozilla.org/r/154510/#review159578

Comment 5

[Rail Aliiev [:rail]]

Comment on attachment 8883571 [details]
Bug 1378252 - Bump funsize version

remote:   https://hg.mozilla.org/build/puppet/rev/fbb2e545b1ab468416beb1adc9da53bd065fd7bd
remote:   https://hg.mozilla.org/build/puppet/rev/639e8b9f155361d7362598f8b614e7373c0568fb

Comment 6

[Rail Aliiev [:rail]]

I pushed the gecko patch to date and see that it added the funsize routes to the repackage-signing:
https://tools.taskcluster.net/groups/F1l5rESnQuyq_dfByEkGIw/tasks/d35sjkN8Qy-i_Vi8S-i2VQ/details

Comment 7

[Justin Wood (:Callek)]

Comment on attachment 8883425 [details]
Bug 1378252 - Add funsize routes to mac repackage-signing tasks

https://reviewboard.mozilla.org/r/154326/#review159640

Comment 8

[Rail Aliiev [:rail]]

Comment on attachment 8883425 [details]
Bug 1378252 - Add funsize routes to mac repackage-signing tasks

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/154326/diff/1-2/

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/901825f49b23
Add funsize routes to mac repackage-signing tasks r=Callek

Comment 10

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/901825f49b23

Comment 11

[Rail Aliiev [:rail]]

Founds some issues with the approach in funsize:


Jul 06 05:46:53 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler 2017-07-06 05:46:53,019 - funsize.worker - INFO - Looking for gecko revision in [u'CVtHJjsxQ-KfB9gNO8ZYAA', u'Pr9OcxSqQlOjbytRDpHd2g', u'A5DOS37gQY-LF8my_o1uog', u'Z10ewJaRQVqwH3Wi0bIKsA', u'OHtVs_KZRq2--lrJbc2biw']
Jul 06 05:46:53 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler 2017-07-06 05:46:53,298 - funsize.worker - INFO - skipping CVtHJjsxQ-KfB9gNO8ZYAA
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler 2017-07-06 05:46:54,777 - funsize.worker - ERROR - Failed to process message
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler Traceback (most recent call last):
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 343, in process_message
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler     self.dispatch_message(body, message)
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 367, in dispatch_message
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler     gdata = parse_taskcluster_message(message.payload)
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 110, in parse_taskcluster_message
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler     'artifacts'] if 'balrog_props.json' in a['name'])
Jul 06 05:46:54 buildbot-master94.bb.releng.use1.mozilla.com python: funsize_scheduler StopIteration
Jul 06 05:47:16 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler 2017-07-06 05:47:16,708 - funsize.worker - INFO - Looking for gecko revision in [u'Pr9OcxSqQlOjbytRDpHd2g', u'OHtVs_KZRq2--lrJbc2biw', u'dQ0E0_YlTJW5vqyfkIRXCQ', u'CVtHJjsxQ-KfB9gNO8ZYAA', u'Z10ewJaRQVqwH3Wi0bIKsA']
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler 2017-07-06 05:47:18,682 - funsize.worker - ERROR - Failed to process message
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler Traceback (most recent call last):
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 343, in process_message
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     self.dispatch_message(body, message)
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 367, in dispatch_message
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     gdata = parse_taskcluster_message(message.payload)
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 110, in parse_taskcluster_message
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     'artifacts'] if 'balrog_props.json' in a['name'])
Jul 06 05:47:18 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler StopIteration

Comment 12

[Rail Aliiev [:rail]]

Looks like GECKO_HEAD_REV is not enough by itself, the task artifacts also should contain balrog_props.json as well

Comment 13

[Rail Aliiev [:rail]]

Attachment: Funsize scheduler PR #2

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8883935 [details] [review]
Funsize scheduler PR #2

r+ed in the PR

Comment 15

[Rail Aliiev [:rail]]

remote:   https://hg.mozilla.org/build/puppet/rev/6046a99499ae578045d18ddc309f9f4980d8b683
remote:   https://hg.mozilla.org/build/puppet/rev/1658e177581e6de35aa698c88043356b569e20de

Comment 16

[Rail Aliiev [:rail]]

Looks like we got some partials!