843464 - mozharness accidentally runs a11y tests instead of plugin tests for the ipcplugins part of mochitest-other

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Phil Ringnalda (:philor)]

Copy/paste: c:\talos-slave\test\build\venv\Scripts\python -u C:\talos-slave\test\build\tests\mochitest/runtests.py --appname=C:\talos-slave\test\build\application\firefox\firefox.exe --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=C:\talos-slave\test\build\symbols --certificate-path=tests/certs --autorun --close-when-done --console-level=INFO --chrome --a11y --setpref=dom.ipc.plugins.enabled=false --setpref=dom.ipc.plugins.enabled.x86_64=false --ipcplugins

If you copy/paste that, with the --chrome --a11y --ipcplugins, a11y will trump ipcplugins, and you'll get a run of the accessibility tests with in-process plugins, rather than a run of the plugin tests with in-process plugins.

On Mac, where we don't do a11y, it's Copy/paste: /Users/cltbld/talos-slave/test/build/venv/bin/python -u /Users/cltbld/talos-slave/test/build/tests/mochitest/runtests.py --appname=/Users/cltbld/talos-slave/test/build/application/FirefoxNightly.app/Contents/MacOS/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-macosx64/1361408096/firefox-22.0a1.en-US.mac.crashreporter-symbols.zip --certificate-path=tests/certs --autorun --close-when-done --console-level=INFO --chrome --setpref=dom.ipc.plugins.enabled=false --setpref=dom.ipc.plugins.enabled.x86_64=false --ipcplugins

which runs mochitest-chrome with in-process plugins instead of running the plugin tests.

Trunk trees are closed, at least until someone authoritatively says we don't care that it's running totally the wrong tests, at which point we can spring the trap on them, and stop running the suite completely.

Comment 1

[Aki Sasaki (not active)]

This doesn't block talos rollout; this is a unittest failure.

Comment 3

[Aki Sasaki (not active)]

Attachment: copy the array instead of reusing the same array

We are running the suites 3x.
But since we're pointing cmd to the same array, we keep appending to the same command.

So first

18:43:13     INFO - Copy/paste: /home/cltbld/talos-slave/test/build/venv/bin/python -u /home/cltbld/talos-slave/test/build/tests/mochitest/runtests.py --appname=/home/cltbld/talos-slave/test/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/1361408096/firefox-22.0a1.en-US.linux-i686.crashreporter-symbols.zip --certificate-path=tests/certs --autorun --close-when-done --console-level=INFO --setpref=webgl.force-enabled=true --chrome

then


18:55:32     INFO - Copy/paste: /home/cltbld/talos-slave/test/build/venv/bin/python -u /home/cltbld/talos-slave/test/build/tests/mochitest/runtests.py --appname=/home/cltbld/talos-slave/test/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/1361408096/firefox-22.0a1.en-US.linux-i686.crashreporter-symbols.zip --certificate-path=tests/certs --autorun --close-when-done --console-level=INFO --setpref=webgl.force-enabled=true --chrome --a11y

then 


18:57:49     INFO - Copy/paste: /home/cltbld/talos-slave/test/build/venv/bin/python -u /home/cltbld/talos-slave/test/build/tests/mochitest/runtests.py --appname=/home/cltbld/talos-slave/test/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/1361408096/firefox-22.0a1.en-US.linux-i686.crashreporter-symbols.zip --certificate-path=tests/certs --autorun --close-when-done --console-level=INFO --setpref=webgl.force-enabled=true --chrome --a11y --setpref=dom.ipc.plugins.enabled=false --setpref=dom.ipc.plugins.enabled.x86_64=false --ipcplugins


If we copy the array before appending, we should stop appending to the previous command.

Comment 4

[Aki Sasaki (not active)]

Comment on attachment 716364 [details] [diff] [review]
copy the array instead of reusing the same array

r=bustage
http://hg.mozilla.org/build/mozharness/rev/7e0db9d8812b

Comment 5

[Aki Sasaki (not active)]

22:21:12     INFO - Copy/paste: /home/cltbld/talos-slave/test/build/venv/bin/python -u /home/cltbld/talos-slave/test/build/tests/mochitest/runtests.py --appname=/home/cltbld/talos-slave/test/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/1361408096/firefox-22.0a1.en-US.linux-i686.crashreporter-symbols.zip --certificate-path=tests/certs --autorun --close-when-done --console-level=INFO --setpref=webgl.force-enabled=true --a11y