Closed Bug 1237182 Opened 8 years ago Closed 6 years ago

Get rid of buildbot specific stuff in mozharness

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: edgar, Assigned: catlee)

References

Details

(Whiteboard: [releng:q22018])

Attachments

(16 files, 3 obsolete files)

59 bytes, text/x-review-board-request
Callek
: review+
catlee
: checked-in+
Details
46 bytes, text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details | Review
46 bytes, text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details | Review
46 bytes, text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details | Review
46 bytes, text/x-phabricator-request
mtabara
: review+
catlee
: checked-in+
Details | Review
46 bytes, text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details | Review
46 bytes, text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details | Review
46 bytes, text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details | Review
2.72 KB, patch
rwood
: review+
catlee
: checked-in+
Details | Diff | Splinter Review
46 bytes, text/x-phabricator-request
tomprince
: review+
sfraser
: review+
Details | Review
46 bytes, text/x-phabricator-request
mtabara
: review+
Details | Review
46 bytes, text/x-phabricator-request
tomprince
: review+
Details | Review
46 bytes, text/x-phabricator-request
tomprince
: review+
Details | Review
46 bytes, text/x-phabricator-request
tomprince
: review+
Callek
: review+
Details | Review
46 bytes, text/x-phabricator-request
mtabara
: review+
Details | Review
46 bytes, text/x-phabricator-request
catlee
: review+
Details | Review
+++ This bug was initially created as a follow-up of Bug #1215084 +++

Quote:
It would be nice if we could fix https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/testbase.py#159 to be taskcluster aware. We'll need to get rid of buildbot specific stuff like this in mozharness anyway.
Depends on: 1209180
Changing the wording slightly, as we'll still need to support buildbot too for the time being.
Summary: Get rid of buildbot specific stuff in mozharness → Abstract out buildbot specific stuff from mozharness core
We no longer need to support buildbot, so we can remove buildbot-specific stuff in mozharness.
Summary: Abstract out buildbot specific stuff from mozharness core → Get rid of buildbot specific stuff in mozharness
Whiteboard: [releng:q22018]
Attachment #8974140 - Flags: feedback?(bugspam.Callek)
Comment on attachment 8974140 [details]
Bug 1237182: Removing unused buildbot support

https://reviewboard.mozilla.org/r/242428/#review248634

Nothing bad stood out to be here.

::: testing/mozharness/mozharness/mozilla/testing/talos.py:207
(Diff revision 1)
> -                # no opts, check for '--geckoProfile' in try message text directly
> -                if self.try_message_has_flag('geckoProfile'):
> -                    self.gecko_profile = True
> -
>          # finally, if gecko_profile is set, we add that to the talos options
>          if self.gecko_profile:

I *think* this branch is NOTREACHED now, but I am not certain.
Attachment #8974140 - Flags: review+
Attachment #8974140 - Flags: feedback?(bugspam.Callek) → feedback+
Keywords: leave-open
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/67d2aa892562
Removing unused buildbot support r=Callek
Assignee: nobody → catlee
Attachment #8974776 - Attachment is obsolete: true
Backed out changeset 67d2aa892562 (bug 1237182) for breaking Decision task. CLOSED TREE

Log:
https://treeherder.mozilla.org/logviewer.html#?job_id=177910441&repo=autoland&lineNumber=1705
 File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/util/schema.py", line 31, in validate_schema
[task 2018-05-10T17:30:26.402Z]     raise Exception('\n'.join(msg) + '\n' + pprint.pformat(obj))
[task 2018-05-10T17:30:26.402Z] Exception: In test 'raptor-firefox-tp6':
[task 2018-05-10T17:30:26.402Z] extra keys not allowed @ data['mozharness']['no-read-buildbot-config']
[task 2018-05-10T17:30:26.402Z] {u'allow-software-gl-layers': False,
[task 2018-05-10T17:30:26.402Z]  u'build-attributes': {u'always_target': False,
[task 2018-05-10T17:30:26.403Z]                        u'build_platform': u'macosx64',
[task 2018-05-10T17:30:26.403Z]                        u'build_type': u'opt',
[task 2018-05-10T17:30:26.403Z]                        u'kind': u'build',
[task 2018-05-10T17:30:26.403Z]                        u'run_on_projects': [u'all'],
[task 2018-05-10T17:30:26.403Z]                        u'shipping_phase': None,
[task 2018-05-10T17:30:26.403Z]                        u'shipping_product': None},
[task 2018-05-10T17:30:26.403Z]  u'build-label': u'build-macosx64/opt',
[task 2018-05-10T17:30:26.403Z]  u'build-platform': 'macosx64/opt',
[task 2018-05-10T17:30:26.403Z]  u'checkout': False,
[task 2018-05-10T17:30:26.403Z]  u'chunks': 1,
[task 2018-05-10T17:30:26.403Z]  'description': 'Raptor Firefox tp6',
[task 2018-05-10T17:30:26.403Z]  u'docker-image': {u'in-tree': u'desktop1604-test'},
[task 2018-05-10T17:30:26.403Z]  u'e10s': True,
[task 2018-05-10T17:30:26.403Z]  u'instance-size': u'default',
[task 2018-05-10T17:30:26.403Z]  u'job-from': 'raptor.yml',
[task 2018-05-10T17:30:26.403Z]  u'loopback-audio': False,
[task 2018-05-10T17:30:26.403Z]  u'loopback-video': False,
[task 2018-05-10T17:30:26.403Z]  'max-run-time': 1200,
[task 2018-05-10T17:30:26.403Z]  'mozharness': {u'chunked': False,
[task 2018-05-10T17:30:26.403Z]                 u'chunking-args': u'this-chunk',
[task 2018-05-10T17:30:26.404Z]                 'config': {'by-test-platform': {'default': ['raptor/linux_config.py'],
[task 2018-05-10T17:30:26.404Z]                                                 'linux64-ccov/opt': ['raptor/linux64_config_taskcluster.py'],
[task 2018-05-10T17:30:26.404Z]                                                 'macosx.*': ['raptor/mac_config.py'],
[task 2018-05-10T17:30:26.404Z]                                                 'windows.*': ['raptor/windows_config.py'],
[task 2018-05-10T17:30:26.404Z]                                                 'windows10-64-ccov/debug': ['raptor/windows_vm_config.py']}},
[task 2018-05-10T17:30:26.404Z]                 'extra-options': ['--test=raptor-firefox-tp6'],
[task 2018-05-10T17:30:26.404Z]                 'no-read-buildbot-config': True,
[task 2018-05-10T17:30:26.404Z]                 u'requires-signed-builds': False,
[task 2018-05-10T17:30:26.404Z]                 'script': 'raptor_script.py',
[task 2018-05-10T17:30:26.404Z]                 u'set-moz-node-path': True,
[task 2018-05-10T17:30:26.404Z]                 u'tooltool-downloads': False},
[task 2018-05-10T17:30:26.404Z]  u'os-groups': [],
[task 2018-05-10T17:30:26.405Z]  u'reboot': False,
[task 2018-05-10T17:30:26.405Z]  'run-on-projects': {'by-test-platform': {'.*-qr/.*': ['try'],
[task 2018-05-10T17:30:26.405Z]                                           'default': ['try']}},
[task 2018-05-10T17:30:26.405Z]  'suite': 'raptor',
[task 2018-05-10T17:30:26.405Z]  u'test-name': 'raptor-firefox-tp6',
[task 2018-05-10T17:30:26.405Z]  u'test-platform': 'macosx64/opt',
[task 2018-05-10T17:30:26.405Z]  'tier': {'by-test-platform': {'default': 3,
[task 2018-05-10T17:30:26.405Z]                                'linux64-ccov/.*': 3,
[task 2018-05-10T17:30:26.405Z]                                'windows10-64-ccov/.*': 3}},
[task 2018-05-10T17:30:26.405Z]  'treeherder-symbol': 'Rap(tp6)',
[task 2018-05-10T17:30:26.405Z]  'try-name': 'raptor-firefox-tp6',
[task 2018-05-10T17:30:26.405Z]  'virtualization': {'by-test-platform': {'default': 'hardware',
[task 2018-05-10T17:30:26.405Z]                                          'windows10-64-ccov/.*': 'virtual'}},
[task 2018-05-10T17:30:26.405Z]  u'webrender': False}
[taskcluster 2018-05-10 17:30:26.787Z] === Task Finished ===

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=67d2aa892562cb25bff4a7707b016f20560873a3&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified&selectedJob=177910441

Backout:
https://hg.mozilla.org/integration/autoland/rev/9b3ba34a7a1a77c3d9487c557d13e8691879f877
Flags: needinfo?(catlee)
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ba91fec47e90
Removing unused buildbot support r=Callek
Pushed by acelists@atlas.sk:
https://hg.mozilla.org/comm-central/rev/d288b2be548c
Port bug 1237182 - remove buildbot specific stuff. rs=bustage-fix
Backed out for breaking the Windows 2012 Nightlies.
"IOError: Can't find buildprops.json in ['.', 'Z:\\task_1526126483\\build\\src\\testing\\mozharness\\configs']!"

Backout link: https://hg.mozilla.org/mozilla-central/rev/df5dae2daa1b920a3c94e2647a7bd8c2b4a3d48e
Log link: https://treeherder.mozilla.org/logviewer.html#?job_id=178176514&repo=mozilla-central&lineNumber=576
C-C backout:
https://hg.mozilla.org/comm-central/rev/2095ab9585866cef1902d32ddcc9137faff6de51
Backed out changeset d288b2be548c: Port of bug 1237182 which was backed out from M-C. a=jorgk
Comment on attachment 8974140 [details]
Bug 1237182: Removing unused buildbot support

https://reviewboard.mozilla.org/r/242428/#review250182

catlee asked me to look at the interdiff (4-5) and it looks good to me.
Attachment #8974140 - Flags: review+
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2f6fc4ca009e
Removing unused buildbot support r=Callek
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/2b3ad9ae9b0d
Port bug 1237182 - Remove unused buildbot support. rs=bustage-fix
Flags: needinfo?(catlee)
MozReview-Commit-ID: GN7lGzWwcvx
MozReview-Commit-ID: GN7lGzWwcvx
Do we also need to remove the references to read_buildbot_config ?
<https://searchfox.org/mozilla-central/search?q=read_buildbot_config>
Flags: needinfo?(catlee)
(In reply to Bob Clary [:bc:] from comment #23)
> Do we also need to remove the references to read_buildbot_config ?
> <https://searchfox.org/mozilla-central/search?q=read_buildbot_config>

Yes, I've got other patches in the works that do that.
Flags: needinfo?(catlee)
Attachment #8976666 - Attachment is obsolete: true
Comment on attachment 8976663 [details]
Bug 1237182: Remove unused mapper mixin

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1305
Attachment #8976663 - Flags: review+
Attachment #8974140 - Flags: checked-in+
Attachment #8976663 - Flags: checked-in+
Comment on attachment 8977056 [details]
Bug 1237182: Get rid of taskcluster helpers

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1313
Attachment #8977056 - Flags: review+
Comment on attachment 8977057 [details]
Bug 1237182: remove mock(chroot) support

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1314
Attachment #8977057 - Flags: review+
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/0bc0174b2d41
Get rid of taskcluster helpers r=Callek
Attachment #8977057 - Flags: checked-in+
Attachment #8977056 - Flags: checked-in+
Comment on attachment 8977058 [details]
Bug 1237182: removing some blobber references

Mihai Tabara [:mtabara]⌚️GMT has approved the revision.

https://phabricator.services.mozilla.com/D1315
Attachment #8977058 - Flags: review+
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/mozilla-central/rev/3fc29973e494
Fix android l10n nightlies a=bustage
Attachment #8977076 - Attachment description: Bug 1237182: remove purge mixin r=Callek → Bug 1237182: remove mozharness purge mixin r=Callek
Attachment #8979668 - Attachment is obsolete: true
Comment on attachment 8977076 [details]
Bug 1237182: remove mozharness purge mixin r=Callek

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1318
Attachment #8977076 - Flags: review+
Attachment #8977058 - Flags: checked-in+
Attachment #8977076 - Flags: checked-in+
Comment on attachment 8980077 [details]
Bug 1237182: Remove more buildbot references from mozharness r=Callek

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1372
Attachment #8980077 - Flags: review+
Comment on attachment 8980076 [details]
Bug 1237182: Remove BuildbotMixin in favour of AutomationMixin r=Callek

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1371
Attachment #8980076 - Flags: review+
Attached patch raptor.diffSplinter Review
Attachment #8980642 - Flags: review?(rwood)
Comment on attachment 8980642 [details] [diff] [review]
raptor.diff

Review of attachment 8980642 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM and I see your raptor try push is green, thank you!
Attachment #8980642 - Flags: review?(rwood) → review+
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5563ed803d7f
Remove more buildbot references from mozharness r=Callek
https://hg.mozilla.org/integration/mozilla-inbound/rev/c53ad5b24efd
Remove BuildbotMixin in favour of AutomationMixin r=Callek
https://hg.mozilla.org/integration/mozilla-inbound/rev/d52176ab1ec9
remove buildbot_config from raptor mixin r=rwood
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9bfce79fc419
Fix python unitests r=bustage on a CLOSED TREE
Comment on attachment 8981458 [details]
Bug 1237182: Remove set_property(..., write_to_file) support r=tomprince,Callek

Tom Prince [:tomprince] has approved the revision.

https://phabricator.services.mozilla.com/D1447
Attachment #8981458 - Flags: review+
Attachment #8980076 - Flags: checked-in+
Attachment #8980077 - Flags: checked-in+
Attachment #8980642 - Flags: checked-in+
Comment on attachment 8981455 [details]
Bug 1237182: get rid of oauth.txt

Mihai Tabara [:mtabara]⌚️GMT has approved the revision.

https://phabricator.services.mozilla.com/D1444
Attachment #8981455 - Flags: review+
Comment on attachment 8981456 [details]
Bug 1237182: Clean up codecoverage property helpers

Tom Prince [:tomprince] has approved the revision.

https://phabricator.services.mozilla.com/D1445
Attachment #8981456 - Flags: review+
Comment on attachment 8982202 [details]
Bug 1237182: Remove default_blob_upload_servers r=mtabara

Mihai Tabara [:mtabara]⌚️GMT has approved the revision.

https://phabricator.services.mozilla.com/D1474
Attachment #8982202 - Flags: review+
Comment on attachment 8982372 [details]
Bug 1237182: [mozharness] Remove some obsolete solaris related configuration; r?catlee

Chris AtLee [:catlee] has approved the revision.

https://phabricator.services.mozilla.com/D1482
Attachment #8982372 - Flags: review+
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/b36a59489f4d
[mozharness] Remove some obsolete solaris related configuration; r=catlee
Comment on attachment 8981454 [details]
Bug 1237182: Get rid of buildprops.json

Tom Prince [:tomprince] has approved the revision.

https://phabricator.services.mozilla.com/D1443
Attachment #8981454 - Flags: review+
Comment on attachment 8981454 [details]
Bug 1237182: Get rid of buildprops.json

Simon Fraser [:sfraser] ⌚️GMT has approved the revision.

https://phabricator.services.mozilla.com/D1443
Attachment #8981454 - Flags: review+
Comment on attachment 8981458 [details]
Bug 1237182: Remove set_property(..., write_to_file) support r=tomprince,Callek

Justin Wood (:Callek) has approved the revision.

https://phabricator.services.mozilla.com/D1447
Attachment #8981458 - Flags: review+
Comment on attachment 8981457 [details]
Bug 1237182: Remove unused try message handling r=tomprince

Tom Prince [:tomprince] has approved the revision.

https://phabricator.services.mozilla.com/D1446
Attachment #8981457 - Flags: review+
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dab1fa529286
Get rid of buildprops.json r=tomprince,sfraser
https://hg.mozilla.org/integration/mozilla-inbound/rev/de5086b44a78
get rid of oauth.txt r=mtabara
https://hg.mozilla.org/integration/mozilla-inbound/rev/76667272ca81
Clean up codecoverage property helpers r=tomprince
https://hg.mozilla.org/integration/mozilla-inbound/rev/9061381462ac
Remove unused try message handling r=tomprince
https://hg.mozilla.org/integration/mozilla-inbound/rev/feda34d6ae78
Remove set_property(..., write_to_file) support r=tomprince,Callek
https://hg.mozilla.org/integration/mozilla-inbound/rev/3b9d55ec3055
Remove default_blob_upload_servers r=mtabara
The bulk of the buildbot specific code has been removed from mozharness now. I'm calling this done \o/
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Blocks: 1201085
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: