1748914 - [perfdocs] `./mach lint -l perfdocs --fix` fails on windows

Status: RESOLVED DUPLICATE of bug 1764609

(Testing :: Performance, defect, P2)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

./mach lint -l perfdocs --fix
 0:26.59 INFO Found 6 perfdocs directories in ['c:\\Users\\elvis\\mozilla-central\\devtools\\perfdocs', 'c:\\Users\\elvis\\mozilla-central\\python\\mozperftest\\perfdocs', 'c:\\Users\\elvis\\mozilla-central\\testing\\awsy\\perfdocs', 'c:\\Users\\elvis\\mozilla-central\\testing\\fxrecord\\perfdocs', 'c:\\Users\\elvis\\mozilla-central\\testing\\raptor\\raptor\\perfdocs', 'c:\\Users\\elvis\\mozilla-central\\testing\\talos\\perfdocs']
Traceback (most recent call last):
  File "c:\users\elvis\mozilla-central\python\mozlint\mozlint\roller.py", line 71, in _run_worker
    res = func(paths, config, **lintargs)
  File "c:\users\elvis\mozilla-central\python\mozlint\mozlint\types.py", line 57, in __call__
    return self._lint(paths, config, **lintargs)
  File "c:\users\elvis\mozilla-central\python\mozlint\mozlint\types.py", line 196, in _lint
    func(files, config, logger, **lintargs)
  File "c:\Users\elvis\mozilla-central\tools\lint\perfdocs\__init__.py", line 13, in lint
    return perfdocs.run_perfdocs(config, logger=logger, paths=paths, generate=fix)
  File "c:\Users\elvis\mozilla-central\tools\lint\perfdocs\perfdocs.py", line 91, in run_perfdocs
    generator.generate_perfdocs()
  File "c:\Users\elvis\mozilla-central\tools\lint\perfdocs\generator.py", line 244, in generate_perfdocs
    perfdocs_tmpdir = self._create_perfdocs()
  File "c:\Users\elvis\mozilla-central\tools\lint\perfdocs\generator.py", line 152, in _create_perfdocs
    os.path.join(perfdocs_tmpdir, framework_name),
  File "c:\Users\elvis\mozilla-central\tools\lint\perfdocs\utils.py", line 25, in save_file
    f.write(file_content)
  File "c:\mozilla-build\python3\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u274c' in position 2240: character maps to <undefined>
A failure occurred in the perfdocs linter.
✖ 1 problem (0 errors, 0 warnings, 1 failure, 0 fixed)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: WIP: Bug 1748914 - add encoding to outputfile while running `./mach perfdocs`

Did you want to run any of these commands instead: perftest, doc?

Run |mach help| to show a list of commands.. r=#perftest-reviewers

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I am not 100% sure this is the right solution- while this allowed me to run ./mach lint -l perfdocs --fix, I still ended up with failures on try:

0:08.50 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.50 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.50 INFO - TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.50 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.50 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.50 INFO - TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.50 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.50 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.50 INFO + TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.50 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.50 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.50 INFO + TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.60 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.60 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.60 INFO - TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.60 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.60 INFO - TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.60 INFO - TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.60 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.60 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.60 INFO + TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
0:08.60 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
0:08.60 INFO + TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
0:08.60 INFO + TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/python/mozperftest/perfdocs/config.yml:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/python/mozperftest/perfdocs/index.rst:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/awsy/perfdocs/config.yaml:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/awsy/perfdocs/index.rst:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/fxrecord/perfdocs/config.yml:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/fxrecord/perfdocs/index.rst:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/raptor/raptor/perfdocs/config.yml:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/raptor/raptor/perfdocs/index.rst:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/talos/perfdocs/config.yml:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.)
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/testing/talos/perfdocs/index.rst:0 | PerfDocs are outdated, run ./mach lint -l perfdocs --fix` to update them. (Flawless performance docs.) 

Comment 3

[Jari Jalkanen]

It seems like addind encoding="utf-8" to all the open()-calls in the utils.py file helps with the issue.

Comment 4

[Greg Mierzwinski [:sparky]]

Jarl, if you want to provide a patch with a fix I'd be happy to review it.

Comment 5

[Greg Mierzwinski [:sparky]]

Perfdocs should no longer have any issues when running on windows after the patch in bug 1764609 lands.