Closed Bug 524775 Opened 10 years ago Closed 8 years ago

Enable a11y and tests on OSX debug builds on accessibility branch

Categories

(Release Engineering :: General, defect, P2)

x86
macOS
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: catlee, Assigned: armenzg)

References

Details

(Whiteboard: [10.6][unittest])

Attachments

(6 files, 4 obsolete files)

3.87 KB, patch
nthomas
: review+
Details | Diff | Splinter Review
19.98 KB, patch
nthomas
: review+
Details | Diff | Splinter Review
1.42 KB, text/plain
nthomas
: review-
Details
11.77 KB, patch
lsblakk
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
4.46 KB, patch
davidb
: review+
catlee
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
34.01 KB, patch
catlee
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
No description provided.
Blocks: 507540
Component: Release Engineering → Release Engineering: Future
Assignee: nobody → catlee
Component: Release Engineering: Future → Release Engineering
Duplicate of this bug: 540465
Assignee: catlee → nobody
Priority: -- → P5
Can the priority on this bug be bumped?
OS: Linux → Mac OS X
That depends on how much there is to do here, and how important it is compared to other work as we approach the end of the quarter. I'm pretty sure we won't be able to get it this month.

Note that accessibility is still disabled by default on mac, and we don't test it on opt builds either. If we knew that mochitest-a11y ran green on m-c opt and debug builds that would help a lot.
I'm aware of one failure in test_combobox.xul which I can disable (by sniffing Mac in the test file).
Assignee: nobody → catlee
Priority: P5 → P3
OK, landed 551548. Tests currently run green on Mac.
This, combined with the other patch, results in enabling a11y tests on macosx-debug on mozilla-central, places, tracemonkey, and addons.  It actually disables a11y tests on the refcounting electrolysis builds, but those are going away soon anyway.

I found this snippet helpful to add at the bottom of master?.cfg:

import re
for b in c['builders']:
    print b['name'], b['factory'].__class__.__name__
    for s in b['factory'].steps:
        step_class, args = s
        args = re.sub(' (instance )?at 0x[0-9a-f]+>', '>', str(args))
        print "   ", step_class.__name__, args

then buildbot checkconfig spits out the list of factories and steps, and you can compare the two.
Attachment #432939 - Flags: review?(nrthomas)
(In reply to comment #7)
> Created an attachment (id=432939) [details]
> This, combined with the other patch, results in enabling a11y tests on
> macosx-debug on mozilla-central, places, tracemonkey, and addons.

Does this automagically include try server?
(In reply to comment #8)
> (In reply to comment #7)
> > Created an attachment (id=432939) [details] [details]
> > This, combined with the other patch, results in enabling a11y tests on
> > macosx-debug on mozilla-central, places, tracemonkey, and addons.
> 
> Does this automagically include try server?

It will, once bug 520227 lands.
Attachment #432937 - Flags: review?(nrthomas) → review+
Comment on attachment 432939 [details] [diff] [review]
Move a11y test flag into platform section of config.py, and enable for macosx-debug most branches

>diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
>             },
>+            'enable_a11y_tests': False,
>         },
>         'win32': {

Overall this patch looks fine to me, but why is a11y False on opt builds ? Shouldn't we restore it to what we had for the refcounting builds ?

mozconfig patch to follow to set 'ac_add_options --enable-accessibility' on opt and debug builds mac ?
Attachment #432939 - Flags: review?(nrthomas) → review+
(In reply to comment #10)
> (From update of attachment 432939 [details] [diff] [review])
> >diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
> >             },
> >+            'enable_a11y_tests': False,
> >         },
> >         'win32': {
> 
> Overall this patch looks fine to me, but why is a11y False on opt builds ?
> Shouldn't we restore it to what we had for the refcounting builds ?
> 
> mozconfig patch to follow to set 'ac_add_options --enable-accessibility' on opt
> and debug builds mac ?

My understanding is that we don't want accessibility enabled in the nightly builds.  Maybe David Bolter can confirm this.
(In reply to comment #11)
> (In reply to comment #10)
> > (From update of attachment 432939 [details] [diff] [review] [details])
> > >diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
> > >             },
> > >+            'enable_a11y_tests': False,
> > >         },
> > >         'win32': {
> > 
> > Overall this patch looks fine to me, but why is a11y False on opt builds ?
> > Shouldn't we restore it to what we had for the refcounting builds ?
> > 
> > mozconfig patch to follow to set 'ac_add_options --enable-accessibility' on opt
> > and debug builds mac ?
> 
> My understanding is that we don't want accessibility enabled in the nightly
> builds.  Maybe David Bolter can confirm this.

Confirmed. Our Mac a11y is not useful at this time due to VoiceOver interop issues.
Attachment #433329 - Flags: review?(nrthomas)
Comment on attachment 433329 [details]
Add enable-accessibility to osx debug builds

IIRC the plan was to break the symlink on the 1.9.1 configs so that we don't enable accessibility there.
Attachment #433329 - Flags: review?(nrthomas) → review-
Depends on: 548768
Haven't had time to work on this in a while, back in the pool!
Assignee: catlee → nobody
Whiteboard: [10.6][unittest]
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
Priority: P3 → P4
Assignee: armenzg → bear
Whiteboard: [10.6][unittest] → [10.6][unittest][triageoldbugs]
Whiteboard: [10.6][unittest][triageoldbugs] → [10.6][unittest]
I can enable a11y for 10.5 on my work on bug 581213 and bug 580410. I am going to be tackling these 2 bugs immediately.

Can I proceed and enable it? Who could have a look at the results once it is running on staging?
(In reply to comment #16)
> I can enable a11y for 10.5 on my work on bug 581213 and bug 580410. I am going
> to be tackling these 2 bugs immediately.
> 
> Can I proceed and enable it? Who could have a look at the results once it is
> running on staging?

David, can you do this?
From the staging run a11y does not even start:
INFO | runtests.py | Server pid: 342
INFO | runtests.py | Websocket server pid: 343
INFO | runtests.py | Running tests: start.

pk12util: PKCS12 IMPORT SUCCESSFUL
INFO | automation.py | SSL tunnel pid: 351
INFO | automation.py | Application pid: 352
TEST-UNEXPECTED-FAIL | automation.py | application timed out after 330 seconds with no output
Can't trigger Breakpad, just killing process
INFO | automation.py | Application ran for: 0:05:30.016987
INFO | automation.py | Reading PID log: /var/folders/Xr/Xr--yJnSEY0U11ET5NZuMU+++TM/-Tmp-/tmpmDvP6kpidlog
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!

10.5's log:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1281973061.1281974055.23598.gz
10.6's log:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1281971865.1281972944.18468.gz

If I have time I will give it a shot to run it manually. If the output is the same I will not be enabling a11y test suite for mac.

From the logs is there anything I am doing wrong? any special environment variable that should be added?
Assignee: bear → nobody
Priority: P4 → P3
Whiteboard: [10.6][unittest] → [10.6][unittest][triagefollowup]
Found in triage.

(In reply to comment #17)
> (In reply to comment #16)
> > I can enable a11y for 10.5 on my work on bug 581213 and bug 580410. I am going
> > to be tackling these 2 bugs immediately.
> > 
> > Can I proceed and enable it? Who could have a look at the results once it is
> > running on staging?
> 
> David, can you do this?

David/surkov: 

1) who is the dev. contact to verify these tests before we enable them? 

2) sanity check: we do already run these tests on opt linux, linux64, win32 builds. However, comment#12 makes me wonder if these tests are of any use on mac?
I'm your guy. Armen, just grab me and I'll check your setup.
Taking.

Can we look at it in a couple of weeks? There is just a whole load on me.

If not, we will look for someone who can take this before me.
Assignee: nobody → armenzg
Priority: P3 → P4
Whiteboard: [10.6][unittest][triagefollowup] → [10.6][unittest]
OK it has been a couple of weeks, and this bug is becoming a strong want.
Time to get to it.
Priority: P4 → P2
FWIW, if we're not planning to ship those bits to our users, we should probably have two types of Mac builds on Mac (which would obviously suck, but we need to run tests on the same bits which we would ship to users anyways.)
As per in-person discussion, let's do this on a new accessibility branch. Filed as bug 649351.
Depends on: 649351
To be clear, this bug is still valid and needed, but depends on the branch bug now.
Let's file a new bug for the trunk work when we get there then.
Summary: Enable a11y and tests on OSX debug builds → Enable a11y and tests on OSX debug builds on accessibility branch
OK so next we need the accessibility branch nightlies to have accessibility built in.

Adding this to mozconfig is how we do it locally:
ac_add_options --enable-accessibility
davidb, could you land a customized mozconfig on your repo? [1]

It probably should be called mozconfig-extra-macosx64 with the extra line you mention.

This is the generic one that is being used:
http://hg.mozilla.org/build/buildbot-configs/raw-file/7bf87d4f860d/mozilla2/macosx64/generic/nightly/mozconfig

[1] https://wiki.mozilla.org/DisposableProjectBranches#Using_a_custom_mozconfig
Yeah I need to figure out how to make sure the extra mozconfig never gets merged to central.
OK, I went ahead and added the extra mozconfig:
http://hg.mozilla.org/projects/accessibility/rev/3a7ec7ea5b78

... but I don't think it is working. Can you find out if the enable-accessibility option is hardcoded somewhere for mac builds?
(In reply to comment #33)
> OK, I went ahead and added the extra mozconfig:
> http://hg.mozilla.org/projects/accessibility/rev/3a7ec7ea5b78
> 
> ... but I don't think it is working. Can you find out if the
> enable-accessibility option is hardcoded somewhere for mac builds?

It might need a clobber first.
(In reply to comment #34)
> (In reply to comment #33)
> > OK, I went ahead and added the extra mozconfig:
> > http://hg.mozilla.org/projects/accessibility/rev/3a7ec7ea5b78
> > 
> > ... but I don't think it is working. Can you find out if the
> > enable-accessibility option is hardcoded somewhere for mac builds?
> 
> It might need a clobber first.

Actually, it does seem to be enabled after all :)
(I need to update my accessibility diagnostic addon)
Besides adding a11y tests for mac I am doing some clean up.
I know we could optimize DEFAULT_TALOS_VALUES but mixing those values on SUITES was making this patch too big.
Attachment #525145 - Attachment is obsolete: true
Attachment #531337 - Flags: review?(lsblakk)
Comment on attachment 531337 [details] [diff] [review]
[buildbot-configs] add ability to add test suites for project_branches plus some cleanup

>diff --git a/mozilla-tests/config.py b/mozilla-tests/config.py
>--- a/mozilla-tests/config.py
>+++ b/mozilla-tests/config.py
>@@ -136,16 +136,41 @@ ALL_PLATFORMS = PLATFORMS['linux']['slav
>                 PLATFORMS['macosx64']['slave_platforms']
> 
> NO_WIN = PLATFORMS['macosx64']['slave_platforms'] + PLATFORMS['linux']['slave_platforms'] + PLATFORMS['linux64']['slave_platforms']
> 
> NO_MAC = PLATFORMS['linux']['slave_platforms'] + PLATFORMS['linux64']['slave_platforms'] + PLATFORMS['win32']['slave_platforms'] + PLATFORMS['win64']['slave_platforms']
> 
> ANDROID = PLATFORMS['android']['slave_platforms']
> 
>+DEFAULT_TALOS_VALUES = {
>+    'chrome':               (True, {}, ALL_PLATFORMS),
>+    'nochrome':             (True, {}, ALL_PLATFORMS),
>+    'dirty':                (True, TALOS_DIRTY_OPTS, ALL_PLATFORMS),
>+    'tp4':                  (True, TALOS_TP4_OPTS, ALL_PLATFORMS),
>+    'cold':                 (True, TALOS_DIRTY_OPTS, NO_WIN),
>+    'v8':                   (True, {}, ALL_PLATFORMS),
>+    'svg':                  (True, {}, ALL_PLATFORMS),
>+    'scroll':               (True, {}, ALL_PLATFORMS),
>+    'dromaeo':              (True, {}, ALL_PLATFORMS),
>+    'addon':                (False, TALOS_ADDON_OPTS, ALL_PLATFORMS),
>+    'addon-baseline':       (False, TALOS_BASELINE_ADDON_OPTS, ALL_PLATFORMS),
>+    'a11y':                 (True, {}, NO_MAC),
>+    'paint':                (True, {}, ALL_PLATFORMS),
>+    'remote-ts':            (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tdhtml':        (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tsvg':          (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tsspider':      (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tpan':          (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tp4m':          (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tp4m_nochrome': (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-twinopen':      (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+    'remote-tzoom':         (True, TALOS_REMOTE_FENNEC_OPTS, ANDROID),
>+}
>+
> # these three are for mozilla-1.9.1 and mozilla-1.9.2
> OLD_BRANCH_ALL_PLATFORMS = PLATFORMS['linux']['slave_platforms'] + \
>                 PLATFORMS['win32']['slave_platforms'] + \
>                 PLATFORMS['macosx']['slave_platforms']
> 
> OLD_BRANCH_NO_WIN = PLATFORMS['macosx']['slave_platforms'] + PLATFORMS['linux']['slave_platforms']
> 
> OLD_BRANCH_NO_MAC = PLATFORMS['linux']['slave_platforms'] + PLATFORMS['win32']['slave_platforms']
>@@ -219,16 +244,34 @@ def removeSuite(suiteName, suiteList):
>         name, suites = info
>         # Let's see if suiteName is on this list of suites
>         if suiteName in suites:
>             suites = suites[:]
>             suites.remove(suiteName)
>             suiteList[i] = (name, suites)
>     return suiteList
> 
>+def addSuite(suiteGroupName, newSuiteName, suiteList):
>+    # In UNITTEST_SUITES we have opt, debug and mobile unit tests keys.
>+    # Each one of these have a list of tuples of test suites.
>+    #     e.g. suiteGroup = ('reftest', ['reftest])
>+    newSuiteList = []
>+    added = False
>+    for tuple in suiteList:
>+        name, suites = tuple
>+        if suiteGroupName == name:
>+            suites.append(newSuiteName)
>+            added = True
>+        newSuiteList.append((name, suites))

Check your indenting here, a non-match gets added twice ^ and below too.

>+
>+    if not added:
>+        newSuiteList.append((name, suites))
>+
>+    return newSuiteList
>+



> # You must define opt_unittest_suites when enable_opt_unittests is True for a 
> # platform. Likewise debug_unittest_suites for enable_debug_unittests
> PLATFORM_UNITTEST_VARS = {
>         'linux': {
>             'builds_before_reboot': 1,
>             'unittest-env' : {'DISPLAY': ':0'},
>             'enable_opt_unittests': True,
>             'enable_debug_unittests': True,
>@@ -869,61 +912,69 @@ BRANCHES['try']['platforms']['macosx64']
> BRANCHES['try']['platforms']['macosx64']['snowleopard']['opt_unittest_suites'] += [('jetpack', ['jetpack'])]
> BRANCHES['try']['platforms']['macosx64']['leopard']['opt_unittest_suites'] += [('jetpack', ['jetpack'])]
> BRANCHES['try']['platforms']['win32']['xp']['opt_unittest_suites'] += [('jetpack', ['jetpack'])]
> BRANCHES['try']['platforms']['win32']['xp']['debug_unittest_suites'] += [('jetpack', ['jetpack'])]
> BRANCHES['try']['platforms']['win32']['win7']['opt_unittest_suites'] += [('jetpack', ['jetpack'])]
> BRANCHES['try']['platforms']['win32']['win7']['debug_unittest_suites'] += [('jetpack', ['jetpack'])]
> BRANCHES['try']['platforms']['android']['enable_opt_unittests'] = True
> 
>-######## generic branch variables for project branches
>-for branch in ACTIVE_PROJECT_BRANCHES:
>-    branchConfig = PROJECT_BRANCHES[branch]
>+def loadDefaultValues(BRANCHES, branch, branchConfig):

I'd like this to still be called PROJECT_BRANCHES so that it's not confused with something that would run on all branches.

>     BRANCHES[branch]['repo_path'] = branchConfig.get('repo_path', 'projects/' + branch) 
>     BRANCHES[branch]['branch_name'] = branchConfig.get('branch_name', branch.title())
>     BRANCHES[branch]['mobile_branch_name'] = branchConfig.get('mobile_branch_name', branch.title())
>     BRANCHES[branch]['build_branch'] = branchConfig.get('build_branch', branch.title())
>     BRANCHES[branch]['talos_command'] = branchConfig.get('talos_cmd', TALOS_CMD)
>     BRANCHES[branch]['fetch_symbols'] = branchConfig.get('fetch_symbols', True)
>     BRANCHES[branch]['support_url_base'] = branchConfig.get('support_url_base', 'http://build.mozilla.org/talos')
>     BRANCHES[branch]['enable_unittests'] = branchConfig.get('enable_unittests', True)
>     # Check if Talos is enabled, if False, set 0 runs for all suites
>     if branchConfig.get('enable_talos') == False:
>         branchConfig['talos_suites'] = {}
>         for suite in SUITES.keys():
>             branchConfig['talos_suites'][suite]  = 0
>+
>+def loadCustomTalosSuites(DEFAULT_TALOS_VALUES, branchConfig):
>     # Want to turn on/off a talos suite? Set it in the PROJECT_BRANCHES[branch]['talos_suites'] otherwise, defaults below
>     if branchConfig.get('talos_suites'):
>         talosConfig = branchConfig['talos_suites']
>     else:
>+        # This is the default and will make all talosConfig.get(key,0) calls
>+        # to default to 0 a.k.a. disabled suite
>         talosConfig = {}
>+
>     for suite in SUITES.keys():
>-        if suite.startswith('remote-'):
>-            BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 0), True, TALOS_REMOTE_FENNEC_OPTS, ANDROID)
>-        else:
>-            # 0 runs by default
>-            if suite in ('v8', 'addon', 'addon-baseline', 'cold'):
>-                if suite == 'cold':
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 0), True, TALOS_DIRTY_OPTS, ALL_PLATFORMS)
>-                elif suite == 'addon':
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 0), False, TALOS_ADDON_OPTS, ALL_PLATFORMS)
>-                elif suite == 'addon-baseline':
>-                     BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 0), False, TALOS_BASELINE_ADDON_OPTS, ALL_PLATFORMS)
>-                else:
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 0), True, {}, ALL_PLATFORMS)
>-            else:
>-                # default is 1 run
>-                if suite == 'dirty':
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 1), True, TALOS_DIRTY_OPTS, ALL_PLATFORMS)
>-                elif suite == 'tp4':
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 1), True, TALOS_TP4_OPTS, ALL_PLATFORMS)
>-                elif suite == 'a11y':
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 1), True, {}, NO_MAC)
>-                else:
>-                    BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 1), True, {}, ALL_PLATFORMS)
>+        BRANCHES[branch][suite + '_tests'] = (talosConfig.get(suite, 0), ) + DEFAULT_TALOS_VALUES[suite]
>+
>+def loadCustomUnittestSuites(BRANCHES, branch, branchConfig):
>+    # If you want a project branch to have a different set of unit tests you can
>+    # do the following:
>+    #  - add a key called "add_test_suites"
>+    #  - add a tuple for each test suite with the following format:
>+    #      ('OS_nick', 'platform', 'opt|debug', 'new or existing group', 'suite name')
>+    #      e.g. ('macosx64', 'snowleopard', 'debug', 'mochitest-other', 'a11y')
>+    #
>+    # Old way of adding suites but still the same format
>+    #    BRANCHES['mozilla-central']['platforms']['win32']['win7']['debug_unittest_suites'] \
>+    #        += [('jetpack', ['jetpack'])]
>+    #
>+    for suiteToAdd in branchConfig.get('add_test_suites', []):
>+        type = 'opt_unittest_suites' if suiteToAdd[2] == 'opt' else 'debug_unittest_suites'
>+        # 'debug_unittest_suites' or 'opt_unittest_suites' is a list of tuple
>+        # addSuite() modifies that list and returns a new one with the added suite
>+        BRANCHES[branch]['platforms'][suiteToAdd[0]][suiteToAdd[1]][type] = \
>+            addSuite( suiteGroupName=suiteToAdd[3], newSuiteName=suiteToAdd[4],
>+                      suiteList=BRANCHES[branch]['platforms'][suiteToAdd[0]][suiteToAdd[1]][type])

Can you attach a dump_masters output to show that this works?

>+
>+######## generic branch variables for project branches
>+for branch in ACTIVE_PROJECT_BRANCHES:
>+    branchConfig = PROJECT_BRANCHES[branch]
>+    loadDefaultValues(BRANCHES, branch, branchConfig)
>+    loadCustomTalosSuites(DEFAULT_TALOS_VALUES, branchConfig)
>+    loadCustomUnittestSuites(BRANCHES, branch, branchConfig)
> 
> if __name__ == "__main__":
>     import sys, pprint, re
> 
>     class BBPrettyPrinter(pprint.PrettyPrinter):
>         def format(self, object, context, maxlevels, level):
>             if isinstance(object, WithProperties):
>                 return pprint.PrettyPrinter.format(self, object.fmtstring, context, maxlevels, level)
>diff --git a/mozilla/project_branches.py b/mozilla/project_branches.py
>--- a/mozilla/project_branches.py
>+++ b/mozilla/project_branches.py
>@@ -11,16 +11,19 @@ PROJECT_BRANCHES = {
>             'tp4': 0,
>             'chrome': 0,
>             'nochrome': 0,
>             'dromaeo': 0,
>             'svg': 0,
>             'scroll': 0,
>             'paint': 0,
>         },
>+        'add_test_suites': [
>+            ('macosx64', 'snowleopard', 'opt', 'mochitest-other', 'mochitest-a11y'),
>+        ]
>     },
>     'build-system': {
>         'enable_talos': False,
>     },
>     'devtools':{
>         'enable_nightly': True,
>         # need both of these to turn off mobile completely because of key in generic config.py
>         'enable_mobile': False,
Attachment #531337 - Flags: review?(lsblakk) → review-
The only changes I have done is to move the functions way at the top and make changed a variable from 'branch' to 'projectBranch' to make it more obvious.

(In reply to comment #37)
> >+def addSuite(suiteGroupName, newSuiteName, suiteList):
> >+    # In UNITTEST_SUITES we have opt, debug and mobile unit tests keys.
> >+    # Each one of these have a list of tuples of test suites.
> >+    #     e.g. suiteGroup = ('reftest', ['reftest])
> >+    newSuiteList = []
> >+    added = False
> >+    for tuple in suiteList:
> >+        name, suites = tuple
> >+        if suiteGroupName == name:
> >+            suites.append(newSuiteName)
> >+            added = True
> >+        newSuiteList.append((name, suites))
> 
> Check your indenting here, a non-match gets added twice ^ and below too.
> 
> >+
> >+    if not added:
> >+        newSuiteList.append((name, suites))
> >+
> >+    return newSuiteList
> >+
I am looking at the indenting and I don't see it wrong.
If the suite gets added on the loop then we don't execute the following if statement.
If it has not been added then we run the if statement to append it at the end. 

> Can you attach a dump_masters output to show that this works?
For sure :) I will get it tomorrow.
And the new attachment! :)
Attachment #531337 - Attachment is obsolete: true
Attachment #531750 - Flags: review?(lsblakk)
Comment on attachment 531750 [details] [diff] [review]
[buildbot-configs] add ability to add test suites for project_branches plus some cleanup

I am fixing a couple of things. I will attach the newest in an hour or early tomorrow.
Attachment #531750 - Attachment is obsolete: true
Attachment #531750 - Flags: review?(lsblakk)
davidb, how does this look?

There are hundreds of oranges there.

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1305320283.1305322036.6286.gz&fulltext=1
Current plan (based on discussion with Armen and Raphael):
Investigate cost of building accessibility into our Mac releases, but pref'ed off.
Then for a11y mochitests the pref would be flipped on. Armen is that last part already easily doable with our test infrastructure?
(In reply to comment #42)
> Then for a11y mochitests the pref would be flipped on. Armen is that last
> part already easily doable with our test infrastructure?
Easily doable. It has been done several times for other teams.
I used dump_master.py to verify that only a11y gets added to mochitest-other on the a11y branch for both opt and debug builds.

I know it is a big patch but it is pretty much a no-op + ability to add unit tests from project branches.

I could optimize this patch even more but I feel that this step can be taken and let it sink without bit rotting other people's work.
Attachment #536996 - Flags: review?(lsblakk)
(In reply to comment #42)
> Current plan (based on discussion with Armen and Raphael):
> Investigate cost of building accessibility into our Mac releases, but
> pref'ed off.
> Then for a11y mochitests the pref would be flipped on. Armen is that last
> part already easily doable with our test infrastructure?

dbolter are you tracking that work on any bug? I would like to add it to the dependency.
Done. Bug 661740. Thanks for the ping Armen.
Depends on: 661740
Priority: P2 → P3
Comment on attachment 536996 [details] [diff] [review]
[buildbot-configs] add ability to add test suites for project_branches plus some cleanup

awesome - i'm glad we'll have more control over project branch unittests with this and thanks for the documentation in there as well.
Attachment #536996 - Flags: review?(lsblakk) → review+
Comment on attachment 536996 [details] [diff] [review]
[buildbot-configs] add ability to add test suites for project_branches plus some cleanup

http://hg.mozilla.org/build/buildbot-configs/rev/fb8a29ea0773
http://hg.mozilla.org/build/buildbot-configs/rev/9047ccca3717

I landed adding the ability to add test suites from project_branches from using the feature for accessibility branch.
Just for the sake of making it clear.

I also added "tp" to the list of valid suites.
Attachment #536996 - Flags: checked-in+
We are not releasing the debug bits so it would not matter to get debug builds with accessibility enabled by default.

This gives us the ability to run a11y tests on debug builds.

This is in parallel to bug 661740.
Attachment #539218 - Flags: review?(catlee)
Attachment #539218 - Flags: review?(bolterbugz)
Comment on attachment 539218 [details] [diff] [review]
[mozconfigs] enable accessibility for debug builds

r=me thanks!
Attachment #539218 - Flags: review?(bolterbugz) → review+
Attachment #539218 - Flags: review?(catlee) → review+
Priority: P3 → P2
Comment on attachment 539218 [details] [diff] [review]
[mozconfigs] enable accessibility for debug builds

This has now landed.

The next scheduled reconfigure should get this picked up.
http://hg.mozilla.org/build/buildbot-configs/rev/9b927148d652
Attachment #539218 - Flags: checked-in+
Attached patch add accessibility mozconfigs (obsolete) — Splinter Review
Right now davidb has a mozconfig extra file on his branch to get accessibility enabled but if he merges to mozilla-central he would be enabling it eventually for every branch when he is not quite ready.

Shall we use this patch? or wait for bug 558180?
Attachment #541800 - Flags: feedback?(catlee)
I had added the wrong patch.

I have updated the nightly channel updates and I have left the mobile desktop mozconfigs out as they are not in use.
Attachment #541800 - Attachment is obsolete: true
Attachment #542824 - Flags: review?(catlee)
Attachment #541800 - Flags: feedback?(catlee)
Comment on attachment 542824 [details] [diff] [review]
add accessibility mozconfigs

You need corresponding mozconfigs in mozilla2-staging, right?
Attachment #542824 - Flags: review?(catlee) → review-
(In reply to comment #54)
> Comment on attachment 542824 [details] [diff] [review] [review]
> add accessibility mozconfigs
> 
> You need corresponding mozconfigs in mozilla2-staging, right?

didn't we deprecate those mozconfigs?
(In reply to comment #55)
> (In reply to comment #54)
> > Comment on attachment 542824 [details] [diff] [review] [review] [review]
> > add accessibility mozconfigs
> > 
> > You need corresponding mozconfigs in mozilla2-staging, right?
> 
> didn't we deprecate those mozconfigs?

I believe we stopped using them since we don't have staging masters but preproduction ones (and we use the mozconfigs under mozilla2/ for that).

catlee, r+ under that light?
Comment on attachment 542824 [details] [diff] [review]
add accessibility mozconfigs

yeah, looks fine otherwise.
Attachment #542824 - Flags: review- → review+
Comment on attachment 542824 [details] [diff] [review]
add accessibility mozconfigs

This will be live on the next scheduled reconfig.
http://hg.mozilla.org/build/buildbot-configs/rev/28fbbfce7c17
Attachment #542824 - Flags: checked-in+
Comment on attachment 542824 [details] [diff] [review]
add accessibility mozconfigs

This is now on production:
http://hg.mozilla.org/build/buildbot-configs/rev/2b1e7533a8c1

We now have mozconfig files exclusive to the accessibility project [1], hence, we won't need the mozconfig-extra-macosx64 file anymore [2].

I believe everything on this bug is done.

[1]
armenzg-laptop $ grep -r accessibility mozilla2/*/accessibility
mozilla2/linux/accessibility/nightly/mozconfig:ac_add_options --enable-update-channel=nightly-accessibility
mozilla2/linux/accessibility/nightly-rpm/mozconfig:ac_add_options --enable-update-channel=nightly-accessibility
mozilla2/linux64/accessibility/nightly-rpm/mozconfig:ac_add_options --enable-update-channel=nightly-accessibility
mozilla2/macosx/accessibility/debug/mozconfig:ac_add_options --enable-accessibility
mozilla2/macosx/accessibility/unittest/mozconfig:ac_add_options --enable-accessibility
mozilla2/macosx64/accessibility/debug/mozconfig:ac_add_options --enable-accessibility
mozilla2/macosx64/accessibility/nightly/mozconfig:ac_add_options --enable-update-channel=nightly-accessibility
mozilla2/macosx64/accessibility/nightly/mozconfig:ac_add_options --enable-accessibility

[2] http://hg.mozilla.org/projects/accessibility/file/3a7ec7ea5b78/mozconfig-extra-macosx64
davidb could you strip the mozconfig-extra-macosx64 from your repository?

I belive "hg rm mozconfig-extra-macosx64 && hg commit" should be good enough.
Assignee: armenzg → bolterbugz
Priority: P2 → --
Thanks davidb!
http://hg.mozilla.org/projects/accessibility/rev/201aa2b03a65

I will verify the results in the morning and close this bug.
Assignee: bolterbugz → armenzg
Priority: -- → P2
I see mochitest-other running with a11y.

I see "ac_add_options --enable-accessibility" in both Mac64 logs.

I also see:
cp configs/mozilla2/macosx64/accessibility/debug/mozconfig .mozconfig
cp configs/mozilla2/macosx64/accessibility/nightly/mozconfig .mozconfig

I believe all issues have been dealt with.
You can merge to mozilla-central without pushing any mozconfig changes to it.

Also note that accessibility is enabled in *all* mac debug builds for *all* project branches.
For optimized builds *only* the accessibility branch has accessibility enabled for Mac optimized builds. This could cause problems when merging back to mozilla-central but you also have the ability of pushing first to the try server to double check.

Let me know if you have any questions.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.