Closed Bug 1654915 Opened 10 months ago Closed 8 months ago

[mozinfo] mach xpcshell-test fails with manifestparser.expression.ParseError (on Debian or other distro with non-numeric os version)

Categories

(Testing :: Mozbase, defect, P3)

defect

Tracking

(firefox83 fixed)

RESOLVED FIXED
83 Branch
Tracking Status
firefox83 --- fixed

People

(Reporter: benc, Assigned: benc)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

mach xpcshell-test doesn't seem to work for me at all at the moment.
Full output below.

I'm building Thunderbird.
I get the same result on two different machines, both running Debian.
I've re-run ./mach bootstrap, but didn't seem to help.

I haven't started wading about in python yet, but am happy to do so. I just figured I should check first if there's something simple I've missed.

$ ./mach xpcshell-test
 0:00.11 INFO Found node at /usr/bin/nodejs
 0:00.11 INFO Found moz-http2 at /home/ben/tb/mozilla/testing/xpcshell/moz-http2/moz-http2.js
 0:00.65 INFO Found /home/ben/tb/mozilla/obj-x86_64-pc-linux-gnu/dist/bin/http3server
 0:00.66 INFO Using /home/ben/tb/mozilla/netwerk/test/http3serverDB
Error running mach:

    ['xpcshell-test']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file xpcshell-test| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

manifestparser.expression.ParseError: could not parse: (os == 'mac' && (verify || debug || os_version == '10.14')) || socketprocess_networking
exception: TypeError: '<' not supported between instances of 'str' and 'int'
variables: {'os': 'linux', 'processor': 'x86_64', 'version': 'Debian Gnu/Linux Testing', 'os_version': StringVersion ('Testing'), 'bits': 64, 'has_sandbox': True, 'webrender': False, 'automation': False, 'linux_distro': 'Debian Gnu/Linux', 'appname': 'thunderbird', 'artifact': False, 'asan': False, 'bin_suffix': '', 'buildapp': 'comm/mail', 'buildtype_guess': 'debug', 'cc_type': 'clang', 'ccov': False, 'crashreporter': True, 'datareporting': True, 'debug': True, 'devedition': False, 'early_beta_or_earlier': True, 'healthreport': True, 'mozconfig': '/home/ben/tb/mozilla/mozconfig', 'nightly_build': True, 'normandy': False, 'official': False, 'pgo': False, 'platform_guess': None, 'release_or_beta': False, 'require_signing': False, 'stylo': True, 'sync': False, 'telemetry': False, 'tests_enabled': True, 'toolkit': 'gtk', 'topsrcdir': '/home/ben/tb/mozilla', 'tsan': False, 'ubsan': False, 'updater': True, 'fission': False, 'serviceworker_e10s': True, 'verify': False, 'socketprocess_networking': False}

  File "/home/ben/tb/mozilla/testing/xpcshell/mach_commands.py", line 257, in run_xpcshell_test
    return xpcshell.run_test(**params)
  File "/home/ben/tb/mozilla/testing/xpcshell/mach_commands.py", line 57, in run_test
    return self.run_suite(**kwargs)
  File "/home/ben/tb/mozilla/testing/xpcshell/mach_commands.py", line 43, in run_suite
    return self._run_xpcshell_harness(**kwargs)
  File "/home/ben/tb/mozilla/testing/xpcshell/mach_commands.py", line 130, in _run_xpcshell_harness
    result = xpcshell.runTests(kwargs)
  File "/home/ben/tb/mozilla/testing/xpcshell/runxpcshelltests.py", line 1548, in runTests
    self.buildTestList(options.get('test_tags'), options.get('testPaths'),
  File "/home/ben/tb/mozilla/testing/xpcshell/runxpcshelltests.py", line 994, in buildTestList
    self.alltests = list(map(normalize, mp.active_tests(filters=filters, **mozinfo.info)))
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/manifestparser.py", line 807, in active_tests
    return list(tests)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/filters.py", line 70, in fail_if
    for test in tests:
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/filters.py", line 58, in run_if
    for test in tests:
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/filters.py", line 47, in skip_if
    if tag in test and parse(test[tag], **values):
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 327, in parse
    return ExpressionParser(text, values).parse()
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 312, in parse
    six.reraise(ParseError, ParseError("could not parse: %s\nexception: %svariables: %s" %
  File "/home/ben/tb/mozilla/third_party/python/six/six.py", line 695, in reraise
    raise value.with_traceback(tb)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 308, in parse
    return self.expression()
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 292, in expression
    left = t.nud(self)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 142, in nud
    expr = parser.expression()
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 296, in expression
    left = t.led(self, left)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 126, in led
    right = parser.expression(self.lbp)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 292, in expression
    left = t.nud(self)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 142, in nud
    expr = parser.expression()
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 296, in expression
    left = t.led(self, left)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 134, in led
    right = parser.expression(self.lbp)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 296, in expression
    left = t.led(self, left)
  File "/home/ben/tb/mozilla/testing/mozbase/manifestparser/manifestparser/expression.py", line 77, in led
    return left == parser.expression(self.lbp)
  File "/home/ben/tb/mozilla/testing/mozbase/mozinfo/mozinfo/string_version.py", line 36, in __eq__
    return self.version == self.__to_version(other)
  File "/usr/lib/python3.8/distutils/version.py", line 46, in __eq__
    c = self._cmp(other)
  File "/usr/lib/python3.8/distutils/version.py", line 337, in _cmp
    if self.version < other.version:

I am not able to reproduce, on Ubuntu, for Firefox (mozilla-central): I can run 'mach xpcshell-test' without any errors. The only recent changes I can think of are bug 1639009...but I don't see how that would relate to your stack trace.

The os_version reported in your error looks unusual.

Still having trouble? I think that mozinfo is having trouble understanding your os version.

Please try this and let me know what you get:

$ python3
Python 3.6.9 (default, Jul 17 2020, 12:50:27) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distro import linux_distribution
>>> linux_distribution()
('Ubuntu', '18.04', 'bionic')
>>> 
Flags: needinfo?(benc)

Ahh, that does indeed look like the problem:

$ python3
Python 3.8.3 (default, May 14 2020, 11:03:12) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distro import linux_distribution
>>> linux_distribution()
('Debian GNU/Linux', 'testing', 'bullseye')

Looking through the distro module, I think the "testing" is ultimately coming from lsb_release -a:

$ lsb_release -a 2>/dev/null
Distributor ID:	Debian
Description:	Debian GNU/Linux bullseye/sid
Release:	testing
Codename:	bullseye

I'm not sure if "testing" is a valid value for the Release: field or not. The man page for lsb_release suggests it's supposed to be a version number, which would imply it's a debian issue...
Thanks Geoff - will do some digging.

Flags: needinfo?(benc)

OK, debian version oddities aside, I think it's really this bug: https://bugs.python.org/issue14894

Basically, LooseVersion fails on a lot of cornercases where strings are involved, eg:

>>> from distutils.version import LooseVersion
>>> LooseVersion("1.2.3") == "1.2.3"
True
>>> LooseVersion("1.2beta") == "1.2beta"
True
>>> LooseVersion("1.2.3") == "1.2beta"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/distutils/version.py", line 46, in __eq__
    c = self._cmp(other)
  File "/usr/lib/python3.8/distutils/version.py", line 337, in _cmp
    if self.version < other.version:
TypeError: '<' not supported between instances of 'int' and 'str'
>>> 

But it doesn't look like it'll be fixed. According to one of the comments there it seems LooseVersion isn't really intended for general use, and they don't want to risk breaking anything. Sigh.

I realise I'm probably running a bit of an outlier case here (with a non-numeric os version)... but would you be open to considering a patch replacing StringVersion with code which doesn't use LooseVersion?

Flags: needinfo?(gbrown)

Here's a revised StringVersion implementation which works for me (and lets me run xpcshell-test).
Removes the reliance on distutils version.LooseString (as per advice in https://bugs.python.org/issue14894#msg328056 ).

Doesn't really guarantee any ordering rules when used on non-numeric version strings, but equality checks behave as expected and it shouldn't crash when presented with odd version strings.

If this general approach looks acceptable (and doesn't break anything else!) let me know and I can submit it formally through phabricator.

Flags: needinfo?(gbrown)
Attachment #9167007 - Flags: feedback?(gbrown)
Comment on attachment 9167007 [details] [diff] [review]
1654915-string-tolerant-version-parsing-1.patch

Review of attachment 9167007 [details] [diff] [review]:
-----------------------------------------------------------------

I like removing the reliance on LooseString. I haven't closely reviewed this, but at a high level it looks good. Thanks!
Attachment #9167007 - Flags: feedback?(gbrown) → feedback+

I tried running a xpcshell test on ArchLinux, and got the following error:

manifestparser.expression.ParseError: could not parse: (os == 'mac' && (verify || debug || os_version == '10.14')) || socketprocess_networking
exception: TypeError: '<' not supported between instances of 'str' and 'int'
variables: {'os': 'linux', 'processor': 'x86_64', 'version': 'Arch Linux Rolling', 'os_version': StringVersion ('Rolling'),  ...
$ python3 -c 'import distro;print(distro.linux_distribution())'
('Arch Linux', 'rolling', 'n/a')

(In reply to Rob Wu [:robwu] from comment #7)

$ python3 -c 'import distro;print(distro.linux_distribution())'
('Arch Linux', 'rolling', 'n/a')

Ahh, so not just Debian then. Good to know.
I shall submit my patch for a proper review.

Assignee: nobody → benc
Status: NEW → ASSIGNED
Attachment #9167007 - Attachment is obsolete: true
Severity: -- → S3
Component: XPCShell Harness → Mozbase
Priority: -- → P3
Summary: mach xpcshell-test fails with manifestparser.expression.ParseError → [mozinfo] mach xpcshell-test fails with manifestparser.expression.ParseError (on Debian or other distro with non-numeric os version)
See Also: → 1622789

I started to hit this bug myself when I try to run TB's xpcshell-tests (I use Debian. )
It took me a while to update my local patches to work with the recent directory layout change.

Funny, though, I could successfully run xpcshell-test of TB on June 16 and June 22.
I check mochitest more often and so didn't realize this bug has crept in.

What has changed?

my system python seems to be version 2 variety.

ishikawa@ip030:/NREF-COMM-CENTRAL/mozilla$ which python
/usr/bin/python
ishikawa@ip030:/NREF-COMM-CENTRAL/mozilla$ python --version
Python 2.7.18

whereas TB build process uses

 checking for Python 3... /NEW-SSD/moz-obj-dir/objdir-tb3/_virtualenvs/init_py3/bin/python (3.8.5)

But I just checked that python version 2 variety system binary is used for invoking xpcshell test locally.

Hmm. So maybe python2 vs python3 thingy as noted by some comments in the patch in bug 1622789.

Any idea when Ben's patch is installed soon?

TIA

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:benc, could you have a look please?
For more information, please visit auto_nag documentation.

Flags: needinfo?(benc)

Whoops - didn't notice the activity in Phabricator! Added a checkin-needed tag there.
Thanks Geoff!

Flags: needinfo?(benc)
Pushed by rmaries@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1d39d7d60ceb
Make mozinfo version parsing tolerant of non-numeric version strings. r=gbrown
Status: ASSIGNED → RESOLVED
Closed: 8 months ago
Resolution: --- → FIXED
Target Milestone: --- → 83 Branch
You need to log in before you can comment on or make changes to this bug.