Last Comment Bug 1181261 - Remove concept of in-tree mozharness configs; merge existing in-tree configs with normal ones
: Remove concept of in-tree mozharness configs; merge existing in-tree configs ...
Status: RESOLVED FIXED
:
Product: Release Engineering
Classification: Other
Component: Mozharness (show other bugs)
: unspecified
: Unspecified Unspecified
-- normal (vote)
: ---
Assigned To: Andrew Halberstadt [:ahal]
: Jordan Lund (:jlund)
:
Mentors:
Depends on: 1131856 1188330 1205934
Blocks:
  Show dependency treegraph
 
Reported: 2015-07-07 11:41 PDT by Andrew Halberstadt [:ahal]
Modified: 2015-09-23 13:19 PDT (History)
11 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
fixed
affected

MozReview Requests
Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:
Show discarded requests

Attachments
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester (40 bytes, text/x-review-board-request)
2015-08-17 11:58 PDT, Andrew Halberstadt [:ahal]
cmanchester: review+
Details | Review
Bustage fix for hidden Mnw tests (1.76 KB, patch)
2015-08-25 06:47 PDT, Andrew Halberstadt [:ahal]
ahalberstadt: review+
Details | Diff | Splinter Review
MozReview Request: Bug 1181261 - Remove remaining in-tree mozhanress configs. r?ahal (40 bytes, text/x-review-board-request)
2015-09-20 19:52 PDT, Edgar Chen [:edgar][:echen]
ahalberstadt: review+
Details | Review

Description User image Andrew Halberstadt [:ahal] 2015-07-07 11:41:28 PDT
Mozharness itself and all its configs will soon be in-tree. Once that happens, there will be no point in maintaining separate "in-tree" mozharness configs (all configs will be in-tree!). We should merge the current in-tree configs with the normal configs, and remove any logic related to them from mozharness core and various scripts.
Comment 1 User image Andrew Halberstadt [:ahal] 2015-08-17 11:58:46 PDT
Created attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

Bug 1181261 - Remove in-tree configs from mozharness, r=chmanchester
Comment 2 User image Andrew Halberstadt [:ahal] 2015-08-17 12:20:25 PDT
Here's a working try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6c39d80aa410

Doesn't seem like I missed anything.. but there's so many configs, it's hard to tell. Sorry for the large patch! (Most of it is just copy/paste from one file to another)
Comment 3 User image Chris Manchester (:chmanchester) 2015-08-19 14:16:10 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

https://reviewboard.mozilla.org/r/16315/#review14813

Lots of great cleanup in here!

::: testing/mozharness/mozharness/mozilla/testing/testbase.py:446
(Diff revision 1)
> -    def _read_tree_config(self):
> +    def _read_try_config(self):

No big deal, but we could move this to mozharness too if we wanted.
Comment 4 User image Andrew Halberstadt [:ahal] 2015-08-20 06:40:59 PDT
https://reviewboard.mozilla.org/r/16315/#review14813

> No big deal, but we could move this to mozharness too if we wanted.

Good idea, might as well and then we can delete the config/mozharness directory entirely.
Comment 5 User image Andrew Halberstadt [:ahal] 2015-08-20 07:09:17 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

The config files under testing/config/mozharness were created so that certain mozharness options
such as test harness arguments could ride the trees, simplifying a lot of logic in mozharness.
But now that mozharness itself is in-tree, these configs no longer serve any purpose. Instead
they are merged into the main configs at testing/mozharness/configs.
Comment 6 User image Andrew Halberstadt [:ahal] 2015-08-20 07:12:16 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

Hmm, I can't figure out how to get mozreview to re-flag you, so doing it here. You should probably take a look at the second diff on the slider. I moved try_arguments.py into TryToolsMixin.
Comment 7 User image Andrew Halberstadt [:ahal] 2015-08-20 07:58:57 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

The config files under testing/config/mozharness were created so that certain mozharness options
such as test harness arguments could ride the trees, simplifying a lot of logic in mozharness.
But now that mozharness itself is in-tree, these configs no longer serve any purpose. Instead
they are merged into the main configs at testing/mozharness/configs.
Comment 8 User image Andrew Halberstadt [:ahal] 2015-08-20 08:00:37 PDT
Oops, try run uncovered bustage:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=2080990ce070

Pushed fix, here's a new try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e78c25206fdc
Comment 9 User image Chris Manchester (:chmanchester) 2015-08-20 11:54:01 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

https://reviewboard.mozilla.org/r/16315/#review14925

Much better, thank you for doing that.
Comment 10 User image Andrew Halberstadt [:ahal] 2015-08-21 07:35:15 PDT
Not really sure why, but jobs didn't get scheduled on that last try run :/.. Here's a new one that worked:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0634eb3619d8
Comment 11 User image Andrew Halberstadt [:ahal] 2015-08-21 07:38:19 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

The config files under testing/config/mozharness were created so that certain mozharness options
such as test harness arguments could ride the trees, simplifying a lot of logic in mozharness.
But now that mozharness itself is in-tree, these configs no longer serve any purpose. Instead
they are merged into the main configs at testing/mozharness/configs.
Comment 12 User image Andrew Halberstadt [:ahal] 2015-08-21 07:40:55 PDT
Latest push also removes testing/config/mozharness/taskcluster_linux_config.py. I talked to :garndt, and it wasn't being used anymore.

It also re-adds the if statement that makes sure this code only runs on try. Oops, that would have been bad. I don't think this requires re-review (and the try push above has these changes included).
Comment 15 User image Andrew Halberstadt [:ahal] 2015-08-21 08:48:27 PDT
Looks like mulet tests don't get run with try: -a:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6c39d80aa410

Hooray..
Comment 16 User image Andrew Halberstadt [:ahal] 2015-08-21 11:01:16 PDT
Oh! Taskcluster doesn't use in-tree mozharness yet. Ironically the push that makes builds use in-tree mozharness landed three pushes after mine. But it was still broken, so guess we have to do something similar for tests.
Comment 17 User image Andrew Halberstadt [:ahal] 2015-08-24 10:51:01 PDT
Comment on attachment 8648877 [details]
MozReview Request: Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

Bug 1181261 - Merge configs from testing/config/mozharness into mozharness proper, r=chmanchester

The config files under testing/config/mozharness were created so that certain mozharness options
such as test harness arguments could ride the trees, simplifying a lot of logic in mozharness.
But now that mozharness itself is in-tree, these configs no longer serve any purpose. Instead
they are merged into the main configs at testing/mozharness/configs.
Comment 18 User image Andrew Halberstadt [:ahal] 2015-08-24 10:56:32 PDT
Rather than wait for bug 1188330, which I'm not sure has an ETA yet, the above patch simply doesn't delete linux_config.py or linux_mulet_config.py yet. It adds a comment to each saying they aren't being used except for mulet. Here's a try run that proves it fixes the mulet tests:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0250b4ba8933

Once bug 1188330 is fixed, they can be moved. I'll re-use this bug for tackling that.
Comment 20 User image Ryan VanderMeulen [:RyanVM] 2015-08-24 17:47:43 PDT
https://hg.mozilla.org/mozilla-central/rev/473c927692ca
Comment 21 User image Bevis Tseng[:bevistseng][:btseng] 2015-08-25 03:15:24 PDT
Unfortunately, now we get the following error when running the marionette test case in treeherder:

Traceback (most recent call last):
  File "scripts/scripts/marionette.py", line 506, in <module>
    marionetteTest = MarionetteTest()
  File "scripts/scripts/marionette.py", line 174, in __init__
    config={'require_test_zip': True})
  File "/builds/slave/test/scripts/mozharness/base/vcs/vcsbase.py", line 116, in __init__
    super(VCSScript, self).__init__(**kwargs)
  File "/builds/slave/test/scripts/mozharness/base/script.py", line 1473, in __init__
    rw_config = ConfigClass(config_options=config_options, **kwargs)
  File "/builds/slave/test/scripts/mozharness/base/config.py", line 265, in __init__
    self.parse_args(args=option_args)
  File "/builds/slave/test/scripts/mozharness/base/config.py", line 481, in parse_args
    options.config_files + options.opt_config_files, options=options
  File "/builds/slave/test/scripts/mozharness/base/config.py", line 446, in get_cfgs_from_files
    all_cfg_files_and_dicts.append((cf, parse_config_file(cf)))
  File "/builds/slave/test/scripts/mozharness/base/config.py", line 159, in parse_config_file
    execfile(file_path, global_dict, local_dict)
  File "/builds/slave/test/scripts/scripts/../configs/marionette/automation_emulator_config.py", line 44
    "suite_definitions": {
    ^
IndentationError: unexpected indent
program finished with exit code 1

http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-central-emulator/1440464312/mozilla-central_ubuntu64_vm-b2g-emulator_test-marionette-webapi-bm118-tests1-linux64-build37.txt.gz
Comment 22 User image Andrew Halberstadt [:ahal] 2015-08-25 06:47:12 PDT
Created attachment 8652319 [details] [diff] [review]
Bustage fix for hidden Mnw tests

Sorry about that, didn't catch it in the try run as it was hidden. This patch should fix it.
Comment 24 User image Ryan VanderMeulen [:RyanVM] 2015-08-25 14:01:32 PDT
https://hg.mozilla.org/mozilla-central/rev/836b81148458
Comment 25 User image Edgar Chen [:edgar][:echen] 2015-08-25 20:07:03 PDT
(In reply to Andrew Halberstadt [:ahal] from comment #16)
> Oh! Taskcluster doesn't use in-tree mozharness yet. Ironically the push that
> makes builds use in-tree mozharness landed three pushes after mine. But it
> was still broken, so guess we have to do something similar for tests.

Hi :ahal, do you know is there any existing bug tracking on using in-tree mozharness on taskcluster? If no, I'd like to file one for it. Thank you.
Comment 26 User image Andrew Halberstadt [:ahal] 2015-08-25 20:32:11 PDT
The patch for taskcluster builds using in-tree mozharness landed recently. The bug for taskcluster tests using in-tree mozharness is bug 1188330.
Comment 27 User image Josh Cheng [:josh] 2015-08-26 00:25:08 PDT
Hi Andrew,

Bug 1188330 is currently the major blocker for emulator-x86-KK. We do need the fix. 

Thank you!
Comment 28 User image Andrew Halberstadt [:ahal] 2015-08-26 06:17:56 PDT
Hey Josh, I think you have it mixed up. This bug isn't blocking anything. The only work left to do here is remove the two files under testing/config/mozharness once bug 1188330 gets fixed. I'm not sure what you are blocked on, but it certainly isn't this.

As for making taskcluster tests use the in-tree mozharness, I'd ask over in bug 1188330, that's not something that I know how to do.
Comment 29 User image Jordan Lund (:jlund) 2015-08-27 20:11:56 PDT
hey andrew, sorry to make you double check, I noticed that this bug removed config/mozharness/b2g_emulator_config.py. it looks like that config is still being pointed at by something for x86-kk or at least that's what within https://bugzil.la/1188330 suggests. is it possible this config was deleted from testing/config/ and not ported to the in-tree /testing/mozharness ?
Comment 30 User image Josh Cheng [:josh] 2015-08-28 02:12:49 PDT
(In reply to Andrew Halberstadt [:ahal] from comment #28)
> Hey Josh, I think you have it mixed up. This bug isn't blocking anything.
> The only work left to do here is remove the two files under
> testing/config/mozharness once bug 1188330 gets fixed. I'm not sure what you
> are blocked on, but it certainly isn't this.
> 
> As for making taskcluster tests use the in-tree mozharness, I'd ask over in
> bug 1188330, that's not something that I know how to do.

Hi Andrew,
Thanks for the info. I will check with Jonathan for who can help here.
Comment 31 User image Andrew Halberstadt [:ahal] 2015-08-28 06:22:14 PDT
(In reply to Jordan Lund (:jlund) from comment #29)
> hey andrew, sorry to make you double check, I noticed that this bug removed
> config/mozharness/b2g_emulator_config.py. it looks like that config is still
> being pointed at by something for x86-kk or at least that's what within
> https://bugzil.la/1188330 suggests. is it possible this config was deleted
> from testing/config/ and not ported to the in-tree /testing/mozharness ?

Thanks for double checking, it didn't sound like anything was broken from the comments above. It's not that it wasn't ported over, it's that taskcluster tests don't use the in-tree mozharness yet. I'll re-add that file for now.
Comment 33 User image Hsin-Yi Tsai [:hsinyi] 2015-08-31 03:31:59 PDT
(In reply to Andrew Halberstadt [:ahal] from comment #31)
> (In reply to Jordan Lund (:jlund) from comment #29)
> > hey andrew, sorry to make you double check, I noticed that this bug removed
> > config/mozharness/b2g_emulator_config.py. it looks like that config is still
> > being pointed at by something for x86-kk or at least that's what within
> > https://bugzil.la/1188330 suggests. is it possible this config was deleted
> > from testing/config/ and not ported to the in-tree /testing/mozharness ?
> 
> Thanks for double checking, it didn't sound like anything was broken from
> the comments above. It's not that it wasn't ported over, it's that
> taskcluster tests don't use the in-tree mozharness yet. I'll re-add that
> file for now.

Hi Andrew, I still observed the error below happening to marionette and marionette-webapi tests [1] and learnt that your push in comment 32 seemed not cover those two. Could you take a look again?

== error ==
The in-tree configuration file '/home/worker/build/tests/config/mozharness/marionette.py' does not exist!It must be added to 'gecko/testing/config/mozharness/marionette.py'. See bug 1035551 for more details.

[1] https://treeherder.allizom.org/#/jobs?repo=try&revision=d472039e6e52
Comment 35 User image Andrew Halberstadt [:ahal] 2015-08-31 07:06:31 PDT
Re-added marionette.py as well.
Comment 36 User image Ryan VanderMeulen [:RyanVM] 2015-08-31 12:15:58 PDT
https://hg.mozilla.org/mozilla-central/rev/fa68da43bac9
Comment 37 User image Ryan VanderMeulen [:RyanVM] 2015-08-31 20:23:29 PDT
https://hg.mozilla.org/mozilla-central/rev/ca3986781f87
Comment 38 User image Edgar Chen [:edgar][:echen] 2015-09-20 19:52:29 PDT
Created attachment 8663478 [details]
MozReview Request: Bug 1181261 - Remove remaining in-tree mozhanress configs. r?ahal

Bug 1181261 - Remove remaining in-tree mozhanress configs. r?ahal
Comment 40 User image Andrew Halberstadt [:ahal] 2015-09-21 10:22:06 PDT
Comment on attachment 8663478 [details]
MozReview Request: Bug 1181261 - Remove remaining in-tree mozhanress configs. r?ahal

https://reviewboard.mozilla.org/r/19759/#review17861

Thanks for doing this! So that means that taskcluster test jobs are using the in-tree mozharness now?
Comment 41 User image Andrew Halberstadt [:ahal] 2015-09-21 10:23:21 PDT
Your try push missed mulet btw, might want to double check that just in case.
Comment 42 User image Edgar Chen [:edgar][:echen] 2015-09-21 18:54:16 PDT
(In reply to Andrew Halberstadt [:ahal] from comment #40)
> Comment on attachment 8663478 [details]
> MozReview Request: Bug 1181261 - Remove remaining in-tree mozhanress
> configs. r?ahal
> 
> https://reviewboard.mozilla.org/r/19759/#review17861
> 
> Thanks for doing this! So that means that taskcluster test jobs are using
> the in-tree mozharness now?

Yes, now all taskcluster tests are using the in-tree one now (bug 1188330).
Comment 43 User image Edgar Chen [:edgar][:echen] 2015-09-21 18:55:40 PDT
(In reply to Andrew Halberstadt [:ahal] from comment #41)
> Your try push missed mulet btw, might want to double check that just in case.

Try runs with Mulet:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3c1087a8f664&exclusion_profile=false&group_state=expanded
Comment 45 User image Wes Kocher (:KWierso) 2015-09-23 13:19:38 PDT
https://hg.mozilla.org/mozilla-central/rev/ad28fa105c30

Note You need to log in before you can comment on or make changes to this bug.