Closed
Bug 985718
Opened 10 years ago
Closed 10 years ago
Schedule desktop tests only on Holly
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jgriffin, Assigned: billm)
References
(Blocks 1 open bug)
Details
Attachments
(4 files)
3.32 KB,
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
15.80 KB,
text/plain
|
Details | |
727 bytes,
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
1.14 KB,
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
Billm is using the holly twig for some work, and needs only desktop tests scheduled on it; Android and B2G tests should not be scheduled there.
Assignee | ||
Updated•10 years ago
|
Blocks: e10s-tests
Assignee | ||
Comment 1•10 years ago
|
||
Thinking a little more about this, I'd like to restrict the tests to mochitests and reftests. No talos, Cpp, XPCShell, Jetpack, or marionette. Also, it would be nice if I could change this myself. Is there a place I can check code into that will affect the tests being run?
Assignee | ||
Comment 2•10 years ago
|
||
Right now it looks like no tests are being run at all on holly. Hopefully this bug can also fix that.
Comment 3•10 years ago
|
||
You'll need to add a new entry for holly here: http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/config.py#1724 If you end up doing it yourself, definitely get review from someone in releng though :). Though it's strange that nothing is running at all.. there might be something else that needs to be done.
Reporter | ||
Comment 4•10 years ago
|
||
We'll also need to update the tree here: http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/project_branches.py#214
Comment 5•10 years ago
|
||
Ok, first we need to change the builds. http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/project_branches.py#214 Remove the gecko_version override, since you probably want to mimic mozilla-central. You probably want to keep branch_projects at [] to avoid building spidermonkey. Set pgo_strategy to None (turn off pgo builds) Remove the enable_nightly, create_snippet, create_partial lines (updates) Add a 'lock_platforms': True And make the 'platforms' dict look like this: http://hg.mozilla.org/build/buildbot-configs/file/f55936ddc584/mozilla/config.py#l1562 Of course, if you don't care about all those platforms, remove the ones you don't want to run. If you don't want talos, add |'enable_talos': False,| For limiting the tests, you'll do something like we did for Cedar here, except we are adding suites for Cedar: http://hg.mozilla.org/build/buildbot-configs/file/f55936ddc584/mozilla-tests/config.py#l1794 You want to remove suites, but instead of a -= or += you can set each BRANCHES['holly']['platforms'][PLATFORM][SLAVE_PLATFORM]['{opt,debug}_unittest_suites'] to the set of suites you care about. The defaults are specified here: http://hg.mozilla.org/build/buildbot-configs/file/f55936ddc584/mozilla-tests/config.py#l579 The slave platforms are listed in this block: http://hg.mozilla.org/build/buildbot-configs/file/f55936ddc584/mozilla-tests/config.py#l102 https://wiki.mozilla.org/ReleaseEngineering:TestingTechniques has hints on how to test your configs once you edit them. I think you'll have to set up buildbot locally to get this to work, so it won't be doable out of the box, but maybe we can update the docs as we go.
Comment 6•10 years ago
|
||
I think this may work for test-masters.sh, but I hit an error during the pip install step, so may need debugging. mkdir foo cd foo virtualenv venv . venv/bin/activate hg clone http://hg.mozilla.org/build/buildbot hg clone http://hg.mozilla.org/build/buildbot-configs hg clone http://hg.mozilla.org/build/buildbotcustom hg clone http://hg.mozilla.org/build/tools cd buildbot-configs # this next step may need a munged master-pip.txt pip install -r master-pip.txt cd ../buildbot/master hg up -r buildbot-0.8 python setup.py install cd ../../buildbot-configs export PYTHONPATH=../..:../../..:../../tools/lib/python:../../../tools/lib/python ./test-masters.sh This worked for me on osx when I applied the below patch; maybe it'll work out of the box for you: (venv)deathduck:/src/billm/buildbot-configs [16:27:26] (default) 900$ hg diff master-pip.txt diff --git a/master-pip.txt b/master-pip.txt --- a/master-pip.txt +++ b/master-pip.txt @@ -1,16 +1,15 @@ Jinja2==2.5.5 MySQL-python==1.2.3 SQLAlchemy==0.6.4 -Twisted==10.1.0 +https://pypi.python.org/packages/source/T/Twisted/Twisted-12.1.0.tar.bz2#md5=f396f1d6f5321e869c2f89b2196a9eb5 amqplib==0.6.1 anyjson==0.3 argparse==1.1 carrot==0.10.7 -distribute==0.6.14 pyOpenSSL==0.10 pyasn1==0.0.11a pycrypto==2.3 pytz==2011d wsgiref==0.1.2 zope.interface==3.6.1 MozillaPulse==0.80
Assignee | ||
Comment 7•10 years ago
|
||
Thanks for all the help, Aki. I've tested this by running mozilla-tests/{b2g-,mobile-,}config.py. For some reason, b2g-config.py didn't even print anything about holly--before or after my patch--but I suspect it worked anyway. I wasn't able to get mozilla/config.py to run properly. There were a couple Python errors. However, from everything you told me, it seems like these changes should be enough. If not, we can try again.
Comment 8•10 years ago
|
||
Comment 9•10 years ago
|
||
Comment on attachment 8394990 [details] [diff] [review] holly-change ># HG changeset patch ># Parent a041bcdc32fde1d050238f6a6564e6eacc629945 ># User Bill McCloskey <wmccloskey@mozilla.com> >diff --git a/mozilla-tests/config.py b/mozilla-tests/config.py >--- a/mozilla-tests/config.py >+++ b/mozilla-tests/config.py >@@ -1796,16 +1796,26 @@ BRANCHES['cedar']['platforms']['macosx64 > BRANCHES['cedar']['platforms']['macosx64']['mavericks']['debug_unittest_suites'] = UNITTEST_SUITES['debug_unittest_suites'][:] > BRANCHES['cedar']['platforms']['win32']['xp-ix']['opt_unittest_suites'] += REFTEST_OMTC[:] > BRANCHES['cedar']['platforms']['win32']['win7-ix']['opt_unittest_suites'] += REFTEST_OMTC[:] > BRANCHES['cedar']['platforms']['win32']['win8']['opt_unittest_suites'] += REFTEST_OMTC[:] > BRANCHES['cedar']['platforms']['win32']['xp-ix']['debug_unittest_suites'] += REFTEST_OMTC[:] > BRANCHES['cedar']['platforms']['win32']['win7-ix']['debug_unittest_suites'] += REFTEST_OMTC[:] > BRANCHES['cedar']['platforms']['win32']['win8']['debug_unittest_suites'] += REFTEST_OMTC[:] > >+# Filter the tests that are enabled on holly. Could you add "for bug 985718" in the comment for posterity? Let me know if the builder list I attached looks good. I think the patch looks good, thanks!
Attachment #8394990 -
Flags: review?(aki) → review+
Assignee | ||
Comment 10•10 years ago
|
||
https://hg.mozilla.org/build/buildbot-configs/rev/bdb5fc2ab72e
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Comment 11•10 years ago
|
||
We tend to leave these open til they've been merged+reconfiged.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Reporter | ||
Comment 12•10 years ago
|
||
Looks like we're scheduling Android tests on Holly. I believe we can (and should, to save resources) turn them off. Can you confirm you don't need Android tests, Bill?
Assignee | ||
Comment 13•10 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #12) > Looks like we're scheduling Android tests on Holly. I believe we can (and > should, to save resources) turn them off. Can you confirm you don't need > Android tests, Bill? That's correct. How do you know we're scheduling them?
Reporter | ||
Comment 14•10 years ago
|
||
Because all the Android tests are showing up at https://tbpl.mozilla.org/?tree=Holly
Assignee | ||
Comment 15•10 years ago
|
||
I've been at a workweek this week, so I haven't done any pushes since this landed. I'll be back tomorrow and do a push. We can see how well it works then :-).
Assignee | ||
Comment 16•10 years ago
|
||
It looks like Android got disabled, but we're still doing b2g builds. https://tbpl.mozilla.org/?tree=Holly&rev=48ded9090d38 What do you think is happening, Aki?
Flags: needinfo?(aki)
Comment 17•10 years ago
|
||
(In reply to Bill McCloskey (:billm) from comment #16) > It looks like Android got disabled, but we're still doing b2g builds. > https://tbpl.mozilla.org/?tree=Holly&rev=48ded9090d38 > > What do you think is happening, Aki? Looks like the 'holly' line in b2g_project_branches needs commenting out. http://hg.mozilla.org/build/buildbot-configs/file/7586f9ebe407/mozilla/b2g_project_branches.py#l67 Could you add a "# disabled for bug 985718" comment above it, as well?
Flags: needinfo?(aki)
Updated•10 years ago
|
Attachment #8398211 -
Flags: review?(aki) → review+
Assignee | ||
Comment 19•10 years ago
|
||
https://hg.mozilla.org/build/buildbot-configs/rev/064ad0c9c07e
Comment 20•10 years ago
|
||
in production.
Assignee | ||
Comment 21•10 years ago
|
||
I'd also like to try to disable non-unified builds. It seems like a big waste of resources. (As an aside, is there any reason to do these on any project branches?)
Attachment #8398669 -
Flags: review?(aki)
Comment 22•10 years ago
|
||
Comment on attachment 8398669 [details] [diff] [review] disable-non-unified Ideally we'd have a way to turn this off per-branch without this loop (if http://hg.mozilla.org/build/buildbotcustom/file/2018f46fa730/misc.py#l1055 checked the branch's enable_nonunified_build setting) but without that check I think this is all you can do. I don't know the reasoning behind this build.
Attachment #8398669 -
Flags: review?(aki) → review+
Assignee | ||
Comment 23•10 years ago
|
||
https://hg.mozilla.org/build/buildbot-configs/rev/f249c447e1af
Comment 24•10 years ago
|
||
in production.
Reporter | ||
Comment 25•10 years ago
|
||
This looks successful. Can we close?
Assignee | ||
Comment 26•10 years ago
|
||
Yes, it looks like this is working. Thanks everyone.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•