1506715 - Adapt Mochitest so it can run on Thunderbird

Status: RESOLVED FIXED

(Testing :: Mochitest, enhancement)

Description

[Geoff Lankow (:darktrojan)]

Like bug 1506523, Mochitest needs a few changes so that Thunderbird can use it. Mostly just using "mail:3pane" where "navigator:browser" is, and avoiding properties that don't exist in a Thunderbird window.

Comment 1

[Geoff Lankow (:darktrojan)]

Attachment: 1506715-mochitest-1.diff

I've also put in the mozharness changes needed for a "thunderbird" suite of tests, so we can avoid running all the others.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 9024581 [details] [diff] [review]
1506715-mochitest-1.diff

Sorry, but I'm not the correct reviewer here. Geoff is a better choice here.

Comment 3

[Geoff Brown [:gbrown]]

:darktrojan - Can you provide some context for me? Are there plans to run other test suites against thunderbird, or just this browser-chrome-thunderbird? Is browser-chrome-thunderbird going to run in continuous integration on mozilla-central, or ...? Why are we doing this now? (I don't know anything about Thunderbird development or testing!)

Comment 4

[Geoff Lankow (:darktrojan)]

We plan to run only the browser-chrome-thunderbird tests on comm-central CI. We don't currently run any mochitests there.

Thunderbird is not in mozilla-central, so it won't be affected.

Comment 5

[Geoff Brown [:gbrown]]

OK, thanks.

I think all the mochitest harness changes are fine: Make mochitest more flexible with respect to the application under test without harming the firefox support. (btw, I suspect we'll need to make similar changes to support geckoview test apps soon).

But I don't understand the mozharness additions. It doesn't seem right to introduce thunderbird configuration alongside all the firefox configurations. Why is it needed? Does comm-central CI use linux_unittest.py, etc?

Comment 6

[Geoff Lankow (:darktrojan)]

We do, and there are two Thunderbird-specific config files in that same directory. I admit mozharness really isn't my area, and I'm just hacking away until it works. Cc'ing Rob, because that's his thing.

Comment 7

[Rob Lemley [:rjl]]

Well, thunderbird_buildbot.py says that once Thunderbird is running on buildbot, that file can basically go away. I'll have to look into that.
As to the *unittest.py changes themselves, if we can figure out a way to add our test to that config["all_mochitest_suites"] list within thunderbird_extra.py, I'm all for it.
Even better (and this is will take some work), we could add support for adding to the Mozharness config path to desktop_unittest.py like we do for fx_desktop_build.py. This would let us move thunderbird_extra.py to comm/mozharness/testing (or something like that).

- Update taskcluster/taskgraph/transforms/tests.py test_description_schema to allow for config-paths key. (See https://hg.mozilla.org/comm-central/file/tip/taskcluster/ci/build/linux.yml#l18)
- Update comm/taskcluster/ci/test/tests.yml with config-path set to "comm/mozharness"
- Update taskcluster/transforms/job/mozharness_test.py to pass --config-path along as MOZHARNESS_CONFIG_PATHS like https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/transforms/job/mozharness.py#l181 
- Update taskcluster/scripts/tester/test-{linux,macosx}.sh to recognize $MOZHARNESS_CONFIG_PATHS env variable like https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/scripts/builder/build-linux.sh#l15 and https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/scripts/builder/build-linux.sh#l89

Maybe more... Like I said, that last part is a lot of work, and probably not necessary at this time.

Comment 8

[Geoff Brown [:gbrown]]

Comment on attachment 9024581 [details] [diff] [review]
1506715-mochitest-1.diff

Review of attachment 9024581 [details] [diff] [review]:
-----------------------------------------------------------------

The mochitest harness changes are fine, but I think you need a clearer separation of thunderbird from the mozharness configuration. Ideally, there shouldn't be thunderbird mozharness configuration in mozilla-central.

Comment 9

[Geoff Lankow (:darktrojan)]

Rob, can you take this from here? I'll tidy up and post the comm-central side of things for you to look at too.

Comment 10

[Rob Lemley [:rjl]]

Attachment: 1506715-mochitest-2.patch

Separating out the mochitest pieces from the mozharness ones.

Comment 11

[Geoff Lankow (:darktrojan)]

Attachment: 1506715-mochitest-3.diff

There's something weird going on with that version 2 patch. Most chunks appear in it twice for some reason.

Could we at least land this part (same as before without the mozharness bits), so that our developers can run tests locally?

Comment 12

[Natalia Csoregi [:nataliaCs]]

Patch failed to apply.
patching file testing/mochitest/browser-test.js
Hunk #5 FAILED at 906
1 out of 5 hunks FAILED -- saving rejects to file testing/mochitest/browser-test.js.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working directory
errors during apply, please fix and qrefresh 1506715-mochitest-3.diff

Comment 13

[Geoff Lankow (:darktrojan)]

Oh damn, I forgot that patch had bitrotted. Will fix.

Comment 14

[Geoff Lankow (:darktrojan)]

Attachment: 1506715-mochitest-4.diff

Let's try that again.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/adcf2d91453e
Adapt Mochitest so it can run on Thunderbird r=darktrojan

Comment 16

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/adcf2d91453e

Comment 17

[Rob Lemley [:rjl]]

Spun off bug 1506715 as the mozharness configuration is turning into it's own task. Geoff, giving this back to you.

Comment 18

[Geoff Lankow (:darktrojan)]

In that case, we're done here.

Comment 19

[Geoff Lankow (:darktrojan)]

I'm pretty sure you meant to say bug 1517298 in that last comment. :)