1800056 - ./mach lint is broken - "mozlog 7.1.0 requires blessings, which is not installed."

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, defect, P2)

Description

[Mark Banner (:standard8)]

On latest mozilla-central on Mac, run:

./mach clobber
./mach lint --setup

or:

./mach clobber
./mach lint -l flake8 python/

The output says mozlog 7.1.0 requires blessings, which is not installed..

https://hg.mozilla.org/integration/autoland/rev/8efafab5495f is the specific revision that caused this, if I back that out, then it works again.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1744340

:ahochheiden, since you are the author of the regressor, bug 1744340, could you take a look? Also, could you set the severity field?

For more information, please visit auto_nag documentation.

Comment 2

[Eemeli Aro [:eemeli]]

For me, mach lint works on macOS, but logging fails on errors during mochitests with something like this:

mozlog.structuredlog: Failure calling log handler:
Traceback (most recent call last):
  File "/Users/eemeli/code/mozilla-unified/third_party/python/blessed/blessed/formatters.py", line 95, in __call__
    attr = curses.tparm(self.encode('latin1'), *args).decode('latin1')
TypeError: an integer is required (got type str)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/structuredlog.py", line 294, in _handle_log
    handler(data)
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/handlers/base.py", line 74, in __call__
    formatted = self.formatter(data)
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/handlers/base.py", line 42, in __call__
    return self.inner(item)
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/formatters/machformatter.py", line 100, in __call__
    s = super(MachFormatter, self).__call__(data)
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/reader.py", line 74, in __call__
    return handler(data)
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/formatters/machformatter.py", line 475, in test_status
    return self._format_status(test, data).rstrip("\n")
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/formatters/machformatter.py", line 182, in _format_status
    rv += self._format_stack(data["stack"])
  File "/Users/eemeli/code/mozilla-unified/testing/mozbase/mozlog/mozlog/formatters/machformatter.py", line 186, in _format_stack
    return "\n%s\n" % self.color_formatter.dim(stack.strip("\n"))
  File "/Users/eemeli/code/mozilla-unified/third_party/python/blessed/blessed/formatters.py", line 101, in __call__
    raise TypeError(
TypeError: Unknown terminal capability, 'dim', or, TypeError for arguments ('SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:497:14\nresult@chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_notificationbox.xhtml:152:18\nrunTimedTests@chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_notificationbox.xhtml:621:22\nrunTimedTestsWait@chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_notificationbox.xhtml:640:18',): an integer is required (got type str)

Comment 3

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Attachment: Bug 1800056 - Change `mozlog` dependency on `blessings` to `blessed` and bump version to `7.1.1` r?ahal

Comment 4

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Attachment: Bug 1800056 - Make `mozlog` and `mozterm` fully compatible with `blessed` r?ahal

• dim does not have an equivalent in Blessed, it will now be grey instead (I visually can't tell the difference, but the ANSI codes are different)
• Some sugar lookups in Blessed (Like normal) don't behave the same as the rest, so our code needed to change to handle the case when the lookup didn't return a callable.
• The tests in test_widgets.py and test_terminal.py were never actually being run (it was skipping because when importing Blessings it failed to import _curses). Given that, a portion of `test_terminal() never actually worked, and it wasn't known because the test didn't run (I didn't just delete half the test case because it didn't work, I deleted it because it can never work =P).
• I added a test_terminal_colors.py just to ensure that there's some coverage there, since dim and normal were failing (but CI was happy).

Depends on D161856

Comment 5

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Sorry about that. This was confusing at first since it looked like there was test coverage (but they weren't failing) Upon closer inspection, there's a pytest.importorskip, but even before blessings was removed it was always being skipped because it couldn't import _ncurses

SKIPPED [1] python\mozterm\test\test_terminal.py:17: could not import 'blessings': No module named '_curses'

That being said, I probably should have done a final searchfox query for blessings to make sure I handled everything. I had no idea we had a whole second logging piece inside mach, but it should work with blessed now. It took me a bit to understand what was causing the problem :eemeli linked above (it was a difference between the implementations of blessings and blessed), but it was a fun little adventure.

I also added some additional test coverage, so that kind of problem shouldn't go unnoticed in the future.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c45689c2f0a1
Make `mozlog` and `mozterm` fully compatible with `blessed` r=firefox-build-system-reviewers,glandium

Comment 7

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/c45689c2f0a1

Comment 8

[msirringhaus]

Attachment: Bug 1800056 - Replace 'dim' in controller/building.py to fix './mach static-analysis check' . r=ahochheiden

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)

Comment 10

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

The version bump for mozlog to 7.1.1 had landed on central. :ahal, could you please release to PyPi when you get a chance?

Comment 11

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

.

Comment 12

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

See bug 1801153 comment 7 for the potential workaround.

I think the PyPi push wasn't necessary to resolve this.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4998287bce99
Replace 'dim' in controller/building.py to fix './mach static-analysis check' . r=ahochheiden

Comment 15

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/4998287bce99