1369425 - Mozscreenshots is not running on Mac OSX

Status: VERIFIED FIXED

(Testing :: mozscreenshots, defect, P1)

Description

[Johann Hofmann [:johannh]]

It doesn't seem to be running in Treeherder and our UI can't get results. https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=692e277e2b9f3bb36d6d67e1166d135228f2d851&filter-tier=1&filter-tier=2&filter-tier=3&exclusion_profile=false&filter-searchStr=ss

Comment from MattN:

09:04:19 <MattN> I think they may have switched bc to TC on OS X so you probably just need to fix https://dxr.mozilla.org/mozilla-central/rev/7b8937970f9ca85db88cb2496f2112175fd847c8/taskcluster/ci/test/tests.yml#623-626
09:05:39 <MattN> probably we can just use `default` instead of specifying platforms

Though I'm not entirely sure how to properly fix and test that. Joel, can you help with that?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

just add a macosx.*: ['mozilla-central', 'try']

to the link mattn provided.  I assume the buildbot code is in place.  

Lastly you need to add the code to the test-platform.yml:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/test/test-platforms.yml#192  (win10 example on line 180)


it should be easy to test on try- if this doesn't work, then I am happy to help dive in and investigate some more.

Comment 2

[Johann Hofmann [:johannh]]

Can you take a look at this failure (and the associated patch)? I haven't been able to figure out why it's not working.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6bf37442b4cde658fb54f076ea2b4d2ff47e851f

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

sorry that you are experiencing this pain, I filed bug 1371315 to document and fix the problem properly and also pushed to try so you can get some data:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6f405a0e64c532d7b31fd62f04733e3b2c4db5a9

Comment 4

[Johann Hofmann [:johannh]]

Thanks! :)

Hm, I can't see the screenshots task on OSX (and it only shows 10.7 and not 10.10) in your try push.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

oh, I found a problem- we were running the screenshots in e10s mode, but in buildbot config they are only defined as non-e10s, so I made a small change (set e10s:false):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=58bc770031d96b4487b661e1fc7935324b87ba8f&selectedJob=105789116

I see the jobs defined in the target tasks uploaded from the gecko decision task, I have higher confidence this will work for you now.

I guess the question is- do these run in e10s mode or only in non-e10s mode?

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

this ran as tier-3:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=58bc770031d96b4487b661e1fc7935324b87ba8f&group_state=expanded&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3&selectedJob=105802782

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: enable browser-screenshots for osx

this assumes a couple things:
1) we only run in non-e10s mode
2) we are ok with osx browser-screenshots running as tier3 (possibly just a treeherder change to make this tier-2)
3) bug 1371315 (patch r?) lands before or with this patch.

Comment 8

[Johann Hofmann [:johannh]]

Comment on attachment 8876209 [details] [diff] [review]
enable browser-screenshots for osx

Redirecting to Matt because I'm not sure about this point:

> 1) we only run in non-e10s mode

Comment 9

[Matthew N. [:MattN]]

(In reply to Johann Hofmann [:johannh] from comment #8)
> Comment on attachment 8876209 [details] [diff] [review]
> enable browser-screenshots for osx
> 
> Redirecting to Matt because I'm not sure about this point:
> 
> > 1) we only run in non-e10s mode

If it will work on OS X in e10s mode then I think that's best. Is there a reason that won't work? We can leave the BB ones on non-e10s for now. The buildbot ones are on non-e10s just because that's what was better tested at the time and we didn't need screenshots of both e10s and non-e10s.

Comment 10

[Matthew N. [:MattN]]

Comment on attachment 8876209 [details] [diff] [review]
enable browser-screenshots for osx

Review of attachment 8876209 [details] [diff] [review]:
-----------------------------------------------------------------

Flag me again if e10s: true isn't possible on OS X for some reason

::: taskcluster/ci/test/tests.yml
@@ +622,5 @@
>      run-on-projects:
>          by-test-platform:
>              windows.*: ['mozilla-central', 'try']
>              linux64/opt: ['mozilla-central', 'try']
> +            macosx.*: ['mozilla-central', 'try']

Note that we only want opt runs, not PGO or debug. The same applies to Windows too.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

the e10s job doesn't run; but this week we are migrating all of OSX to be 100% taskcluster, so if it is ok, I will wait a week and then solve the e10s.

Should I land what I have now and do a follow up to address windows7 to only run on opt as well as the e10s bits, or wait?

Comment 12

[Matthew N. [:MattN]]

I don't really understand what you mean by "the e10s job doesn't run". I'm concerned about switching from e10s to non-e10s for existing platforms where I believe e10s does work now.

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

I only see a screenshots job run, not a screenshots-e10s job running.  As the OSX transition is going on this week, I think early next week we can get ss-e10s running on osx and linux with no problem; possibly win7/win10 if this is something that typically runs inside a VM.

Comment 14

[Matthew N. [:MattN]]

(In reply to Joel Maher ( :jmaher) from comment #13)
> I only see a screenshots job run, not a screenshots-e10s job running.

Are you looking at tier 3? I see it for linux on m-c now at least:

https://cl.ly/3g271O2z000N

Comment 15

[Matthew N. [:MattN]]

I don't see the e10s jobs on your try push[1] but I think that has to do with the try syntax mentioning "mochitest-browser-screenshots" and that implying non-e10s?

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=58bc770031d96b4487b661e1fc7935324b87ba8f&group_state=expanded&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=screen

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

I pushed to try a few days ago:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c34a12d464c04679e3046ef46dd40bd1a94f30d2&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3

and I don't see osx screenshots, and win7 screenshots in non-e10s, this I need to figure out some more- ideally we have ss-e10s running on osx/win7/win10.

Comment 17

[Matthew N. [:MattN]]

Right, I'm saying that if we land the patch (reverting the e10s line) it will still work for mozilla-central maybe, just not Try since the e10s jobs work on m-c now for linux64. Can we give that a try since m-c is more important?

Comment 18

[Joel Maher ( :jmaher ) (UTC -8)]

yes, I will do e10s: both; and see where we get.

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

one final try pass, then will land tonight or early tomorrow.

Comment 20

[Matthew N. [:MattN]]

Sorry, I'm really confused… we only need screenshots on e10s, I think, hence the e10s:true.

On your try push at https://treeherder.mozilla.org/#/jobs?repo=try&revision=c34a12d464c04679e3046ef46dd40bd1a94f30d2&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=screen&group_state=expanded if I click to add jobs I do see the OS X e10s jobs: https://cl.ly/26353o3s3D1p

which I believe means they will run fine on m-c and for `-u all` just not if someone requests `-u mochitest-browser-screenshots`. If that's true then I'm saying we can land that to address the issue for m-c and have a separate bug to fix the try syntax to work with os x.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2d5c7d7cee76
Mozscreenshots is not running on Mac OSX. r=johann

Comment 22

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/2d5c7d7cee76

Comment 23

[Johann Hofmann [:johannh]]

I'm still not seeing screenshots running on OSX.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=a418121d46250f91728b86d9eea331029c264c30&filter-tier=1&filter-tier=2&filter-tier=3&exclusion_profile=false&filter-searchStr=ss

Will needinfo Joel when he comes back from PTO.

Comment 24

[Johann Hofmann [:johannh]]

See comment 23, is there anything we're still missing here?

Comment 25

[Joel Maher ( :jmaher ) (UTC -8)]

I see this is missing, working on a patch for taskcluster- it looks like we have a slight issue of using macosx/opt instead of either:
* macosx64/opt
* macosx.*/opt

same for the windows references

Comment 26

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: fix taskcluster config to have proper platforms to run screenshots

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/aaa4ae876a53
Mozscreenshots is not running on Mac OSX. fix taskcluster config. r=dustin

Comment 28

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/aaa4ae876a53

Comment 29

[Johann Hofmann [:johannh]]

It worked!

Comment 30

[Johann Hofmann [:johannh]]

https://screenshots.mattn.ca/compare/?oldProject=mozilla-central&oldRev=2908b6171cb8df85fc1f0985e2bf94ce3b04a522&newProject=mozilla-central&newRev=9ce83907a2bc83f47e393761b3990432005d8223&filter=osx