1762047 - [manifestparser] skip-if entries can contain conditions with single equals, which evaluate to true

Status: RESOLVED DUPLICATE of bug 1831514

(Testing :: Mozbase, defect)

Description

[Beth Rennie [:beth] (she/her)]

:Gijs noticed that https://bugzilla.mozilla.org/show_bug.cgi?id=1761227 was not showing up as an orange on try and, upon our investigation, we found that the manifest for this test contains:

[browser_Troubleshoot.js]
skip-if = os = "win" && os_version = "6.1" # bug 1715857

and as such this bug is always skipped because it is not doing a comparison (==).

See this treeherder log from m-c

As it turns out there are multiple other instances of this (searchfox query)

	browser/modules/test/browser/browser.ini
8	skip-if = os = "win" && os_version = "6.1" # bug 1715860
	dom/events/test/browser.ini
4	skip-if = (os != 'linux' && os != 'win') || skip-if = !e10s
	toolkit/content/tests/browser/browser.ini
97	skip-if = os == "linux" && bits == 64 && os_version = "18.04" # Bug 1614739
	toolkit/modules/tests/browser/browser.ini
24	skip-if = debug || os = "linux"
26	skip-if = debug || os = "linux"
30	skip-if = os = "win" && os_version = "6.1" # bug 1715857

Comment 1

[Geoff Brown [:gbrown]]

I would expect this to be handled in https://searchfox.org/mozilla-central/source/testing/mozbase/manifestparser/manifestparser/expression.py, but, at a glance, I don't see explicit handling of '='.

Comment 2

[Geoff Brown [:gbrown]]

Keep in mind that manifestparser has "strict" error checking:
https://searchfox.org/mozilla-central/rev/570f6e5c06b6c8140f53bf104d785a18165212ab/testing/mozbase/manifestparser/manifestparser/manifestparser.py#70
which is frequently disabled:
https://searchfox.org/mozilla-central/rev/570f6e5c06b6c8140f53bf104d785a18165212ab/testing/mochitest/runtests.py#1757

(I don't know if that has direct bearing on this issue.)