1918249 - `mach lint` fails to lint prefs JS files on Windows due to charmap encoding issues

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, defect)

Description

[Nick Alexander :nalexander [he/him]]

On Windows, HG revision 50016ed273448d29bc57cb2ca478c95509931435, me and colleagues witness:

$ ./mach lint --fix browser/app/profile/firefox.js
Traceback (most recent call last):
  File "c:\users\nalexander\mozilla\gecko.git\python\mozlint\mozlint\roller.py", line 73, in _run_worker
    res = func(paths, config, **lintargs)
  File "c:\users\nalexander\mozilla\gecko.git\python\mozlint\mozlint\types.py", line 58, in __call__
    return self._lint(paths, config, **lintargs)
  File "c:\users\nalexander\mozilla\gecko.git\python\mozlint\mozlint\types.py", line 149, in _lint
    return func(files, config, **lintargs)
  File "C:\Users\nalexander\Mozilla\gecko.git\tools\lint\libpref\__init__.py", line 110, in checkdupes
    errors.extend(check_against(file, pref_names))
  File "C:\Users\nalexander\Mozilla\gecko.git\tools\lint\libpref\__init__.py", line 54, in check_against
    prefs = read_prefs(path)
  File "C:\Users\nalexander\Mozilla\gecko.git\tools\lint\libpref\__init__.py", line 81, in read_prefs
    for lineno, line in enumerate(source, start=1):
  File "C:\mozilla-build\python3\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 7446: character maps to <undefined>

Running with PYTHONUTF8=1 addresses the issue.

Comment 1

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1918249 - Fix `mach lint` linting prefs JS files on Windows by forcing UTF-8. r?ahal

Prefs files in general are allowed to not be UTF-8, but all the prefs
files in tree are UTF-8, so we can lint them with that assumption.

Comment 2

[Mark Banner (:standard8)]

Is bug 1654395 a duplicate of this?

Comment 3

[Nick Alexander :nalexander [he/him]]

(In reply to Mark Banner (:standard8) from comment #2)

Is bug 1654395 a duplicate of this?

No, but it's similar. (Also: good sleuthing!) This ticket repros outside of my strange environment (Emacs), so it should be addressed with a patch. That one is "just" a weirdness.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ef7f41f546c0
Fix `mach lint` linting prefs JS files on Windows by forcing UTF-8. r=ahal

Comment 5

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/ef7f41f546c0