829513 - Too few tests are running on Mozilla-B2g18 branch

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Currently it seems like we are not running any tests on the Mozilla-B2g18 branch other than the b2g-specific tests.

We should at the very least enable tests on the linux machines since the desktop tests cover a lot of the functionality we are shipping on B2G, and the linux platform is very similar to the B2G one.

But it wouldn't hurt to enable more than that since a lot of our developers are using desktop b2g which shares a lot of code with desktop firefox on all platforms. So if we break desktop gecko it'll hurt our developers.

The linux part here is pretty urgent. The rest is less so so if there are concerns, lets just start with linux.

Comment 1

[Justin Wood (:Callek)]

Per :bmoss in IRC we are to turn on:

* Linux64 Desktop Tests
* OSX Desktop Tests
* Windows Desktop Tests

Full assortment, but explicitly not
* Linux32 Desktop Tests

Comment 2

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #1)
> Per :bmoss in IRC we are to turn on:

> Full assortment, but explicitly not
> * Linux32 Desktop Tests

And he just confirmed against https://bugzilla.mozilla.org/show_bug.cgi?id=818833#c23
That we also want linux32 ipc, but only on b2g18 for now.

Comment 3

[Justin Wood (:Callek)]

Attachment: [configs] part 1 - Turn on most tests

So this does as asked, and discussed with bmoss.

This does not *yet* turn on linux32 ipc (not sure the best patch for that yet, and sicking told me this was more important than ipc for the moment)

This also does not start building+testing android on b2g18, because AIUI thats not a requirement/want atm.

Comment 4

[Justin Wood (:Callek)]

Attachment: tests scheduler dump diff of part 1

This is informative as help to understand what part 1 does "exactly"

Comment 5

[[e-mail] (:bhearsum)]

Comment on attachment 701012 [details] [diff] [review]
[configs] part 1 - Turn on most tests

Review of attachment 701012 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +78,5 @@
>      'mozilla-b2g18':       {
>          'datazilla_url': None,
>          'platforms': {
> +            # desktop per sicking in Bug 829513
> +            'macosx': {},

Why are we turning leopard tests when they're for FF16 and lower? r=me with that dropped and the next hunk removed.

Comment 6

[Justin Wood (:Callek)]

Attachment: [configs] part 2 - turn on ipc stuff

So this is a lot uglier than I'd like, but is jumping through hoops to do the following:
* Manually add every unittest we have that is linux32 ipc
* Since we're leaving linux32 on we want to turn off all other tests
* We turn off Linux32 Talos, which doesn't have a good way to turn off platforms per branch
* Fix addSuite method, which looks like its been broken for a while.

Comment 7

[Justin Wood (:Callek)]

Comment on attachment 701012 [details] [diff] [review]
[configs] part 1 - Turn on most tests

Landed with nit fixed (don't know how I didn't think of just dropping macosx from platforms here)
http://hg.mozilla.org/build/buildbot-configs/rev/d24776ff0678

Comment 8

[Justin Wood (:Callek)]

Attachment: [config] Part 2 - v2

fixes bareword 'name' by accident in here, and the fact that I accidentally extended the tuple length on the talos tests, rather than adding an array back onto the end.

Comment 9

[Justin Wood (:Callek)]

Comment on attachment 701085 [details] [diff] [review]
[config] Part 2 - v2

http://hg.mozilla.org/build/buildbot-configs/rev/794d8de6a774

Comment 10

[[e-mail] (:bhearsum)]

These patches are both in production now.

Comment 11

[Justin Wood (:Callek)]

Attachment: [configs] part 3 - leopard shouldn't be on

Noticed that we still had leopard tests running, I manually hid them for now, but this properly shuts them off.

Comment 12

[Ed Morley [:emorley]]

(Unsurprisingly,) There is orange on desktop b2g18 runs - but with no way to tell what broke them in the prior days/week when tests weren't running. Jonas - can you find someone to take a look?

Comment 13

[Justin Lebar (not reading bugmail)]

I filed bugs and starred the failures.

Thanks a lot for enabling these tests.

Comment 14

[Ed Morley [:emorley]]

Thank you Justin :-)

(Bug 829641 and bug 829637 for anyone wondering)

Comment 15

[Justin Wood (:Callek)]

Attachment: [configs] part 4 - make tests aware that desktop builds are pgo

Comment 16

[[e-mail] (:bhearsum)]

Comment on attachment 701149 [details] [diff] [review]
[configs] part 3 - leopard shouldn't be on

Review of attachment 701149 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry, I misinterpreted this hunk in the original patch.

Comment 17

[[e-mail] (:bhearsum)]

I haven't seen confirmation that this worked yet.

Comment 18

[[e-mail] (:bhearsum)]

OK, I don't see any leopard tests on https://tbpl.mozilla.org/?tree=Mozilla-B2g18&noignore=1&rev=8f2ef4998b60 and I do Cipc/Ripc/Moth.

Comment 19

[Phil Ringnalda (:philor)]

I really doubt you want mochitest-ipcplugins running there - once OOP plugins became the default we switched that suite around, so now it runs the plugin tests with in-process plugins.

Comment 20

[Armen [:armenzg]]

FYI I'm disabling the desktop nightly builds and Windows opt/debug builds for b2g18 and b2g18_v1_1_0hd branches in bug 948135.

I'm commenting here since I saw comments in the code referencing this bug.