932504 - Schedule gaia-ui-tests on b2g desktop builds on osx on cedar, pine

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

Engineering has enabled OOP on b2g desktop builds on OSX.  They'd like to run gaia-ui-tests on those, to make sure we don't break things going forward.

To start, we should schedule these on cedar and pine until we know where we stand.

Comment 1

[Jonathan Griffin (:jgriffin)]

So, bug 917596 claims to have done this, but I don't see the tests on cedar; we can continue the work here, I suppose.

Comment 2

[Jonathan Griffin (:jgriffin)]

I've looked a bit at the buildbot code and can't decipher why the tests aren't showing up on cedar or pine.  The only potential error I see is:

http://hg.mozilla.org/build/buildbot-configs/file/311a1dd1bb26/mozilla-tests/b2g_config.py#l84

(should probably be mac-perf instead of linux-perf?)

...but I doubt that explains why the tests aren't being scheduled.

aki, have any idea what's going on?

Comment 3

[Aki Sasaki (not active)]

Ok, so
https://tbpl.mozilla.org/php/getParsedLog.php?id=29929534&tree=Cedar&full=1 (osx) doesn't have a sendchange, but https://tbpl.mozilla.org/php/getParsedLog.php?id=29930506&tree=Cedar&full=1 (linux64) does.

I think this is because http://hg.mozilla.org/build/buildbot-configs/file/a2cec05c8f08/mozilla/b2g_config.py#l159 sets unittest_masters for linux64_gecko, but http://hg.mozilla.org/build/buildbot-configs/file/a2cec05c8f08/mozilla/b2g_config.py#l232 sets unittest_masters to [] for macosx64_gecko.

You've got an r+ from me to set the macosx64_gecko unittest_masters line to

    'unittest_masters': GLOBAL_VARS['unittest_masters'],

like linux64_gecko; I think that'll fix it.

Comment 4

[Jonathan Griffin (:jgriffin)]

Thanks aki; I now know buildbot a little better than I did before. :)

Comment 5

[Jonathan Griffin (:jgriffin)]

http://hg.mozilla.org/build/buildbot-configs/rev/f1ba644ae1ec

Comment 6

[Jonathan Griffin (:jgriffin)]

Attachment: macosx_gaiaui.diff

Comment 7

[Armen [:armenzg]]

Comment on attachment 825586 [details] [diff] [review]
macosx_gaiaui.diff

This has turned all b2g desktop builds orange.

Buildbot was not in the PATH.

Comment 8

[Aki Sasaki (not active)]

Ugh.
Ok, it's not the mac-perf / linux-perf diff:
http://hg.mozilla.org/build/buildbotcustom/file/242b84c733c0/env.py#l186

Comment 9

[Aki Sasaki (not active)]

Ok, we probably have to tweak the b2g_config.py osx env to look like this:
http://hg.mozilla.org/build/buildbot-configs/file/144a6d68d917/mozilla/config.py#l598

instead of this: http://hg.mozilla.org/build/buildbot-configs/file/144a6d68d917/mozilla/b2g_config.py#l250

probably just /tools/buildbot/bin.  Probably best to test in staging at this point... I'll add it to my list.

Comment 10

[Aki Sasaki (not active)]

Attachment: osx_sendchange.diff

This adds the unittest_masters, adds buildbot to PATH, and fixes some pep8 stuff.
I was able to successfully sendchange on an r5 lion builder in staging.

Comment 11

[Armen [:armenzg]]

Comment on attachment 826208 [details] [diff] [review]
osx_sendchange.diff

Review of attachment 826208 [details] [diff] [review]:
-----------------------------------------------------------------

Those perf environments should not be in use anymore IIUC.

Comment 12

[Aki Sasaki (not active)]

Comment on attachment 826208 [details] [diff] [review]
osx_sendchange.diff

https://hg.mozilla.org/build/buildbot-configs/rev/0e1acbc4533b

Comment 13

[John Hopkins (:jhopkins)]

in production

Comment 14

[Jonathan Griffin (:jgriffin)]

Something still isn't right, as these tests aren't showing up on cedar.   Aki, any ideas?

Comment 15

[Aki Sasaki (not active)]

This bug is cursed?

Digging to see if I have any other clever ideas.

Comment 16

[Aki Sasaki (not active)]

If you look at http://hg.mozilla.org/build/buildbot-configs/rev/0e1acbc4533b#l1.12 , I set the unittest_masters for osx.

If you look at http://hg.mozilla.org/build/buildbot-configs/file/2ff07c61c7a3/mozilla/b2g_config.py#l232 (tip-of-default), unittest_masters is set for osx.

If you look at http://hg.mozilla.org/build/buildbot-configs/file/b6efcd390964/mozilla/b2g_config.py#l232 (tip-of-production), unittest_masters is set to [] for osx.  Something's screwy with the merge here.

Comment 17

[Aki Sasaki (not active)]

When |hg diff -r production| in default, I get no diff for b2g_config.py.
When I copy the default b2g_config.py into production, I get a single line diff, the unittest_masters line.

I think it's official: I hate DVCSes.

Comment 18

[Aki Sasaki (not active)]

Landed http://hg.mozilla.org/build/buildbot-configs/rev/1602af92f93f to try to fix up... will require another reconfig.

Comment 19

[Aki Sasaki (not active)]

Ok, should be live, for real, hopefully.

Comment 20

[Aki Sasaki (not active)]

We have sendchange!

https://tbpl.mozilla.org/php/getParsedLog.php?id=30174319&tree=Cedar&full=1
Executing: ['buildbot', 'sendchange', '--master', 'buildbot-master81.build.mozilla.org:9301', '--username', 'sendchange-unittest', '--branch', 'cedar-macosx64_gecko-unittest', '--revision', 'b9e63daefcec', '--comments', 'Re-enable some b2g desktop mochitests which may have been wrongfully disabled', '--property', 'buildid:20131105142132', '--property', 'pgo_build:False', '--property', 'builduid:670a04c16cfb4580a2cbd974ec97ee02', 'http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/cedar-macosx64_gecko/1383690092/en-US/b2g-28.0a1.en-US.mac64.dmg', 'http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/cedar-macosx64_gecko/1383690092/en-US/b2g-28.0a1.en-US.mac64.tests.zip']
Process stdio:
NOTE: --username/-u is deprecated: use --who/-W'
change sent successfully

program finished with exit code 0


But no tests.  I think I know what's going on...

Comment 21

[Aki Sasaki (not active)]

The production-masters.json has macosx64_gecko set for the osx test masters.
The test builders are there, with mountainlion slaves attached.
The sendchange is successful; it just isn't triggering the tests for some reason, and I'm running out of ideas.

Hoping to figure this out soon, but I'll be out Thurs/Fri and a decent-sized todo list for the rest of today, and this bug has been resistant to fixing.

Comment 22

[Aki Sasaki (not active)]

Attachment: unittest_platform

Ok, I eyeballed the b2g linux64 sendchange (working) and compared it to the macosx64 sendchange (not working), and noticed the branch for linux64 was cedar-linux64_gecko-opt-unittest and the branch for macosx64 was cedar-macosx64_gecko-unittest (no opt).  Manually sendchanging to the macosx64 branch with the opt scheduled a test.

Then I found this line: http://hg.mozilla.org/build/buildbot-configs/file/e641038ee358/mozilla/b2g_config.py#l145 which we're missing for macosx64_gecko.  http://hg.mozilla.org/build/buildbot-configs/file/e641038ee358/mozilla/b2g_config.py#l216

Comment 23

[Armen [:armenzg]]

Comment on attachment 828335 [details] [diff] [review]
unittest_platform

Review of attachment 828335 [details] [diff] [review]:
-----------------------------------------------------------------

I bet that was fun to find.

Comment 24

[Aki Sasaki (not active)]

Comment on attachment 828335 [details] [diff] [review]
unittest_platform

https://hg.mozilla.org/build/buildbot-configs/rev/40cdb4c55c20

Comment 25

[[e-mail] (:bhearsum)]

something here is in production

Comment 26

[Aki Sasaki (not active)]

https://tbpl.mozilla.org/?tree=Cedar&showall=1&jobname=b2g_mtnlion%20cedar%20opt%20test%20gaia-ui-test
!!!

We may need to tweak something to get this to show up in the right row.

Comment 27

[Aki Sasaki (not active)]

Oh, and not try to untar the dmg.

Comment 28

[Jonathan Griffin (:jgriffin)]

Yep, we'll need to generalize the script to not expect a linux environment.

Comment 29

[Jonathan Griffin (:jgriffin)]

Attachment: Use a better name for tests on b2g desktop builds,

This will make tests on Mac B2G desktop builds use a _macosx64 suffix, isntead of _mtnlion.

Comment 30

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 833197 [details] [diff] [review]
Use a better name for tests on b2g desktop builds,

+1 to one less special-casing in TBPL's regexes :)

Comment 31

[Jonathan Griffin (:jgriffin)]

Comment on attachment 833197 [details] [diff] [review]
Use a better name for tests on b2g desktop builds,

https://hg.mozilla.org/build/buildbot-configs/rev/1bc432c630d8

Comment 32

[Justin Wood (:Callek)]

something[s] here made it to production