1585751 - Intermittent /builds/worker/checkouts/gecko/tools/lint/wpt.yml:0 | Lint process exited with return code 2 (wpt)

Status: RESOLVED FIXED

(Testing :: Mochitest, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: dluca [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=269482985&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/QDgnUAQvQBaTB44GsXw4ww/runs/0/artifacts/public/logs/live_backing.log

---

TinderboxPrint:<a href=https://us-west-2.hgmointernal.net/integration/autoland/rev/c10147827a4f8bfcdc1174a609e3a083c8ee4333 title='Built from autoland revision c10147827a4f8bfcdc1174a609e3a083c8ee4333'>c10147827a4f8bfcdc1174a609e3a083c8ee4333</a>
[setup 2019-10-02T18:14:01.201Z] GECKO_PATH is /builds/worker/checkouts/gecko
[task 2019-10-02T18:14:01.201Z] executing ['bash', '-cx', './mach lint -l wpt -f treeherder -f json:/builds/worker/mozlint.json']in /builds/worker/checkouts/gecko
[task 2019-10-02T18:14:01.210Z] + ./mach lint -l wpt -f treeherder -f json:/builds/worker/mozlint.json
[task 2019-10-02T18:14:02.291Z] Using base prefix '/usr'
[task 2019-10-02T18:14:02.291Z] New python executable in /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python3
[task 2019-10-02T18:14:02.292Z] Also creating executable in /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python
[task 2019-10-02T18:14:04.542Z] Installing setuptools, pip, wheel...done.
[task 2019-10-02T18:14:05.819Z] b"running build_ext\nbuilding 'psutil._psutil_linux' extension\ncreating build\ncreating build/temp.linux-x86_64-3.5\ncreating build/temp.linux-x86_64-3.5/psutil\nx86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/include/python3.5m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_common.o\nx86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/include/python3.5m -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_posix.o\nx86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/include/python3.5m -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_linux.o\ncreating build/lib.linux-x86_64-3.5\ncreating build/lib.linux-x86_64-3.5/psutil\nx86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/psutil/_psutil_common.o build/temp.linux-x86_64-3.5/psutil/_psutil_posix.o build/temp.linux-x86_64-3.5/psutil/_psutil_linux.o -o build/lib.linux-x86_64-3.5/psutil/_psutil_linux.cpython-35m-x86_64-linux-gnu.so\nbuilding 'psutil._psutil_posix' extension\nx86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/include/python3.5m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_common.o\nx86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/include/python3.5m -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_posix.o\nx86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/psutil/_psutil_common.o build/temp.linux-x86_64-3.5/psutil/_psutil_posix.o -o build/lib.linux-x86_64-3.5/psutil/_psutil_posix.cpython-35m-x86_64-linux-gnu.so\ncopying build/lib.linux-x86_64-3.5/psutil/_psutil_linux.cpython-35m-x86_64-linux-gnu.so -> psutil\ncopying build/lib.linux-x86_64-3.5/psutil/_psutil_posix.cpython-35m-x86_64-linux-gnu.so -> psutil\n"
[task 2019-10-02T18:14:05.819Z] Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
[task 2019-10-02T18:18:00.147Z] No specific files specified, running the full wpt lint (this is slow)
[task 2019-10-02T18:18:00.147Z] Exception in thread ProcessReader:
[task 2019-10-02T18:18:00.147Z] Traceback (most recent call last):
[task 2019-10-02T18:18:00.147Z]   File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
[task 2019-10-02T18:18:00.147Z]     self.run()
[task 2019-10-02T18:18:00.147Z]   File "/usr/lib/python3.5/threading.py", line 862, in run
[task 2019-10-02T18:18:00.147Z]     self._target(*self._args, **self._kwargs)
[task 2019-10-02T18:18:00.147Z]   File "/builds/worker/checkouts/gecko/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 1085, in _read
[task 2019-10-02T18:18:00.148Z]     callback(line.rstrip())
[task 2019-10-02T18:18:00.148Z]   File "/builds/worker/checkouts/gecko/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 995, in __call__
[task 2019-10-02T18:18:00.148Z]     e(*args, **kwargs)
[task 2019-10-02T18:18:00.148Z]   File "/builds/worker/checkouts/gecko/tools/lint/wpt/wpt.py", line 24, in process_line
[task 2019-10-02T18:18:00.148Z]     data = json.loads(line)
[task 2019-10-02T18:18:00.148Z]   File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
[task 2019-10-02T18:18:00.148Z]     s.__class__.__name__))
[task 2019-10-02T18:18:00.148Z] TypeError: the JSON object must be str, not 'bytes'
[task 2019-10-02T18:18:00.148Z] 
[task 2019-10-02T18:18:00.150Z] TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/tools/lint/wpt.yml:0 | Lint process exited with return code 2 (wpt)
[taskcluster 2019-10-02 18:18:02.636Z] === Task Finished ===
[taskcluster 2019-10-02 18:18:03.525Z] Unsuccessful task run with exit code: 1 completed in 571.919 seconds

Comment 1

[Dorel Luca [:dluca]]

This first happened on this push:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=c10147827a4f8bfcdc1174a609e3a083c8ee4333

When Bug 1585317 landed here:
https://hg.mozilla.org/integration/autoland/rev/c10147827a4f8bfcdc1174a609e3a083c8ee4333

James: Could you please take a look at this bug?

Comment 2

[Andrew Halberstadt [:ahal]]

I'd guess this started happening with bug 1580280.

Comment 3

[Cristina Coroiu [:ccoroiu]]

Fixed by https://hg.mozilla.org/integration/autoland/rev/a49132352f5bfd3994ab179813b6c1be499e7a23

Comment 4

[Emilio Cobos Álvarez (:emilio)]

So this only happens if the lint actually fails.

Comment 5

[Andrew Halberstadt [:ahal]]

I think I see what's happening.

Comment 6

[Andrew Halberstadt [:ahal]]

Couldn't reproduce locally, but verified fixed on try:
https://treeherder.mozilla.org/#/jobs?repo=try&author=ahalberstadt%40mozilla.com&fromchange=5a5407a89dc8ad2adc53bc20fd1b7dbe12d3eae7&tochange=e675e2bf54cebc74a1f2452b39dd2ba1b1173ec1

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1585751 - [lint.wpt] Fix Python 3 error processing 'wpt' output, r?jgraham

Comment 8

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1585751 - [lint.wpt] Work around mozprocess Windows + Python 3 bug, r?jgraham

Depends on D47975

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7cfcbf9b6dfa
[lint.wpt] Fix Python 3 error processing 'wpt' output, r=jgraham
https://hg.mozilla.org/integration/autoland/rev/4ba57765a767
[lint.wpt] Work around mozprocess Windows + Python 3 bug, r=jgraham

Comment 10

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/7cfcbf9b6dfa
https://hg.mozilla.org/mozilla-central/rev/4ba57765a767

Comment 11

[Intermittent Failures Robot]

5 failures in 4292 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 3

Platform breakdown:

• lint: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585751&startday=2019-09-30&endday=2019-10-06&tree=all