1762647 - Support custom Firefox command line arguments in Mochitest manifest files

Status: RESOLVED FIXED

(Testing :: Mochitest, enhancement)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Similar to the prefs entry in browser.ini files as used for browser chrome mochitests it would be great to also be able to specify command line arguments for Firefox. These entries would then apply to all the tests in this particular folder given that one single instance of Firefox is run for all of them.

Currently we have to run workarounds with our tests for CDP (chrome devtools protocol) as implemented via the Remote Agent which is also used for WebDriver BiDi. It would help us a lot if we could just add args = --remote-debugging-protocol to the appropriate browser.ini files instead.

Should mochitests or others also benefit from that?

Andrew or Geoff, your feedback would be welcome. Thanks.

Comment 1

[Andrew Halberstadt [:ahal]]

Can this feature be flipped via environment variable? Those are already supported in manifests. But otherwise I'm not opposed to it.

Heads up that ownership of harnesses has transferred out of Releng to Marco's team (triage owners need updating), so Geoff and I wouldn't be the ones working on this. Test harnesses have always been neglected and they are doubly so now, so you might need to attempt this yourself if it's urgent.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrew Halberstadt [:ahal] from comment #1)

Can this feature be flipped via environment variable? Those are already supported in manifests. But otherwise I'm not opposed to it.

Actually the Remote Agent clearly needs the command line arguments, and that's what we also have to get tested. So environment variables wont work - but still good to know that this can be an option! Thanks.

Heads up that ownership of harnesses has transferred out of Releng to Marco's team (triage owners need updating), so Geoff and I wouldn't be the ones working on this. Test harnesses have always been neglected and they are doubly so now, so you might need to attempt this yourself if it's urgent.

It's not urgent given that we have coverage for that via wpt tests, but I might try to get it implemented if I find the time. And thanks for the info about the upcoming changes.

Comment 3

[Geoff Brown [:gbrown]]

I don't recall anyone wanting command line arguments in their tests before, but if adding for browser tests, I think it would be reasonable to add for all mochitests.

How would you handle platform differences? Android would need special care of course. Are all firefox command line options supported on all desktop platforms?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Geoff Brown [:gbrown] from comment #3)

I don't recall anyone wanting command line arguments in their tests before, but if adding for browser tests, I think it would be reasonable to add for all mochitests.

Yes, so plain mochitests don't use browser.ini but some other manifest file? I assume it has the same layout so that adding it for all of them is exactly the same?

How would you handle platform differences? Android would need special care of course. Are all firefox command line options supported on all desktop platforms?

Yes, for Android we would have to most likely add them to the GeckoView configuration file. Not sure if that is something mochitests already make use of. In geckodriver we use it to start Marionette.

Not all arguments would be supported across all platforms. --foreground does only apply on MacOS. But in general unsupported ones are simply ignored so that shouldn't cause any problem.

Comment 5

[Geoff Brown [:gbrown]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #4)

(In reply to Geoff Brown [:gbrown] from comment #3)

I don't recall anyone wanting command line arguments in their tests before, but if adding for browser tests, I think it would be reasonable to add for all mochitests.

Yes, so plain mochitests don't use browser.ini but some other manifest file? I assume it has the same layout so that adding it for all of them is exactly the same?

Right, mochitest-plain uses mochitest.ini, same layout; should be handled exactly the same.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Geoff Brown [:gbrown] from comment #5)

Right, mochitest-plain uses mochitest.ini, same layout; should be handled exactly the same.

Perfect. So lets get it expanded then.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Bug 1762647 - [mochitest] Add support for custom Firefox command line arguments in manifests.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Geoff, could you please check if the patch looks sane to you? I've verified that it works with my changes on bug 1717899 and the browser chrome tests under remote/cdp/tests/browser.

Are there any other tests I have to add? I don't see that something at the manifestparser level would be needed - can't also find specific tests for preferences. If there is please let me know. Thanks.

Comment 9

[Geoff Brown [:gbrown]]

That looks fine to me, and I'm not aware of prefs tests either.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/24236e43d8f5
[mochitest] Add support for custom Firefox command line arguments in manifests. r=jmaher

Comment 11

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/24236e43d8f5