Closed Bug 1696531 Opened 5 months ago Closed 1 month ago

Run xpcshell-test against Python 3 in CI

Categories

(Firefox Build System :: Task Configuration, task)

task

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ahal, Assigned: bhearsum)

References

(Blocks 2 open bugs)

Details

(Keywords: perf-alert)

Attachments

(10 files, 1 obsolete file)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review

The harness supports Python 3. We should:

  1. Add python-3: true key to all xpcshell tasks
  2. Fix errors that pop up on try as a result
  3. Add testing/xpcshell to the py2 linter's exclude list (so we can start using Python 3 syntax)
Summary: Run reftest/crashtest/jsreftest against Python 3 in CI → Run xpcshell-test against Python 3 in CI
Assignee: nobody → bhearsum
Depends on: 1668903
Duplicate of this bug: 1668903
Attachment #9211509 - Attachment description: WIP: Bug 1696531: fix xpcshell data_test_script.py to support python3 → WIP: Bug 1696531 - [xpcshell] Fix some issues py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js

Hey Jim, I was hoping you could help us out here.

There is one failure left preventing us from switching the xpcshell harness over to Python 3 in CI in netwerk/test/unit/test_unix_domain.js:
https://treeherder.mozilla.org/logviewer?job_id=335085932&repo=try&lineNumber=5723

Basically this test is throwing NS_ERROR_FILE_NAME_TOO_LONG rather than NS_ERROR_FILE_NOT_FOUND on MacOS only.

The only thing that should be changing here is that we're running the harness with Python 3 instead of Python 2. I'm not sure why or how that would cause this failure and I don't have a Mac to reproduce :/. Do you see anything obvious here?

Thanks in advance!

Flags: needinfo?(jimb)

Unix domain sockets have stricter limits on the length of their file names than ordinary files; usually they can't be longer than 107 bytes. If I had to guess, I'd say that the way do_get_tempdir behaves is affected by the change to the harness, and that the Python 2 -> 3 switch is just incidental.

It seems to me that choosing a shorter filename here might work around the problem.

Flags: needinfo?(jimb)

Ah, good guess!

Here's where we set that env:
https://searchfox.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#465

So maybe mkdtemp is a longer path on Python 3 vs 2. I'll poke around a bit next week.

I confirmed that with Python 3 mkdtemp creates a directory that is 2 characters longer than on Python 2. Just verifying the fix on try.

Attachment #9213093 - Attachment description: WIP: Bug 1696531 - [xpcshell] Fix error in selftest.py → Bug 1696531 - [xpcshell] Fix Python 3 error in selftest.py, r?bhearsum
Attachment #9211509 - Attachment description: WIP: Bug 1696531 - [xpcshell] Fix some issues py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js → Bug 1696531 - [xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r?bhearsum

Unix socket names have a fairly short maximum length. Turns out that on Python
3 'mkdtemp' creates directories that have two extra characters as opposed to
Python 2. That was enough for us to hit the max length and break this test.

Work around the issue by using a shorter base name for the socket (which doesn't
make a difference to the test).

Depends on D109730

Attachment #9211506 - Attachment description: WIP: Bug 1696531: use python3 for xpcshell tests → Bug 1696531 - use python3 for xpcshell tests
Attachment #9211507 - Attachment description: WIP: Bug 1696531: fix fake adb process to support python2 and 3 → Bug 1696531: fix fake adb process to support python2 and 3
Attachment #9211510 - Attachment description: WIP: Bug 1696531: fix dmd.py and related tools to support python 3 → Bug 1696531: fix dmd.py and related tools to support python 3
Attachment #9211511 - Attachment description: WIP: Bug 1696531: add required import to xpcshell/selftest.py → Bug 1696531: add required import to xpcshell/selftest.py
Attachment #9211510 - Attachment is obsolete: true
Attachment #9211510 - Attachment is obsolete: false
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a7f9bd32a4c9
fix fake adb process to support python2 and 3 r=ahal
https://hg.mozilla.org/integration/autoland/rev/a8d8adae39b6
fix dmd.py and related tools to support python 3 r=ahal
https://hg.mozilla.org/integration/autoland/rev/0b8c56f2f5c3
add required import to xpcshell/selftest.py r=ahal
https://hg.mozilla.org/integration/autoland/rev/5002c2053444
[xpcshell] Fix Python 3 error in selftest.py, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/1f7ffffa368f
[xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/c907d8324bcc
Use a shorter name in 'test_on_directory' from test_unix_domain.js, r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/52ba6bf74b55
use python3 for xpcshell tests r=bhearsum

Looks like this failure was also hit in bug 1668903 but I didn't notice. I think fix_stacks.py just needs to maintain PY2 compatibility for now as some browser-chrome tests are also using it.

Flags: needinfo?(bhearsum)
Attachment #9211506 - Attachment description: Bug 1696531 - use python3 for xpcshell tests → WIP: Bug 1696531: use python3 for xpcshell tests

This file is also used by some browser-chrome tests which are still Python 2
for now. So let's not drop PY2 compat just yet.

Depends on D109728

Attachment #9211506 - Attachment description: WIP: Bug 1696531: use python3 for xpcshell tests → Bug 1696531: use python3 for xpcshell tests
Attachment #9211506 - Attachment description: Bug 1696531: use python3 for xpcshell tests → WIP: Bug 1696531: use python3 for xpcshell tests
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1f982303513f
fix fake adb process to support python2 and 3 r=ahal
https://hg.mozilla.org/integration/autoland/rev/7dcbb17a1578
fix dmd.py and related tools to support python 3 r=ahal
https://hg.mozilla.org/integration/autoland/rev/33f3e86a5ce8
add required import to xpcshell/selftest.py r=ahal
https://hg.mozilla.org/integration/autoland/rev/dba2bea61b29
[xpcshell] Fix Python 3 error in selftest.py, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/7129c9cfe519
[xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/754ce2bf288a
Use a shorter name in 'test_on_directory' from test_unix_domain.js, r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/c631966a64c0
WIP: Bug 1696531: use python3 for xpcshell tests r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/907102743c5f
Keep Python 2 compatibility in tools/rb/fix_stacks.py for now, r=bhearsum
Attachment #9211511 - Attachment description: Bug 1696531: add required import to xpcshell/selftest.py → Bug 1696531: add required import to xpcshell/selftest.py r=ahal
Attachment #9211510 - Attachment description: Bug 1696531: fix dmd.py and related tools to support python 3 → Bug 1696531: fix dmd.py and related tools to support python 3 r=ahal
Attachment #9211507 - Attachment description: Bug 1696531: fix fake adb process to support python2 and 3 → Bug 1696531: fix fake adb process to support python2 and 3 r=ahal

Backed out 8 changesets (bug 1696531) for test_dmd.js and test_subprocess.js xpc failures.

Push with failures:
https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&test_paths=memory%2Freplace%2Fdmd%2Ftest&fromchange=907102743c5f8c93c8c429dd7c46e603217af1a5&tochange=f31804ec05cf83250dfc897cf0e05730b62b38ae
https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=WAYA3u9qSHOoG9D6dNl7kQ.0&fromchange=907102743c5f8c93c8c429dd7c46e603217af1a5&searchStr=test-verify&tochange=f31804ec05cf83250dfc897cf0e05730b62b38ae

Backout link: https://hg.mozilla.org/integration/autoland/rev/f31804ec05cf83250dfc897cf0e05730b62b38ae

Failures logs:
https://treeherder.mozilla.org/logviewer?job_id=336505051&repo=autoland&lineNumber=2826

[task 2021-04-14T14:26:44.673Z] 14:26:44     INFO -  TEST-START | memory/replace/dmd/test/test_dmd.js
[task 2021-04-14T14:26:46.432Z] 14:26:46  WARNING -  TEST-UNEXPECTED-FAIL | memory/replace/dmd/test/test_dmd.js | xpcshell return code: 0
[task 2021-04-14T14:26:46.432Z] 14:26:46     INFO -  TEST-INFO took 1759ms
[task 2021-04-14T14:26:46.434Z] 14:26:46     INFO -  >>>>>>>
[task 2021-04-14T14:26:46.437Z] 14:26:46     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2021-04-14T14:26:46.438Z] 14:26:46     INFO -  PID 9325 | DMD[9337] $DMD = '1'
[task 2021-04-14T14:26:46.439Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 1 {
[task 2021-04-14T14:26:46.440Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.441Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.442Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.443Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.444Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 2 {
[task 2021-04-14T14:26:46.445Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.450Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 3 {
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 4 {
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Execution measurements {
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that persist after Dump() ends {
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces:         2,496 bytes
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Unused stack traces:       2,704 bytes
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Stack trace table:       196,640 bytes (16,384 entries, 25 used)
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Live block table:         24,608 bytes (512 entries, 30 used)
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Dead block table:             32 bytes (0 entries, 0 used)
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that are destroyed after Dump() ends {
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:         202,888 bytes
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces set:     12,288 bytes
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used PCs set:              12,288 bytes
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Pointer ID map:            20,480 bytes
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Counts {
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:            22 requests
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service cache:   0.0% hit rate, 0.5% occupancy at end
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   }
[task 2021-04-14T14:26:46.451Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.452Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Unknown pointer 0xa
[task 2021-04-14T14:26:46.454Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 5 {
[task 2021-04-14T14:26:46.454Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.454Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.455Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.455Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Execution measurements {
[task 2021-04-14T14:26:46.455Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that persist after Dump() ends {
[task 2021-04-14T14:26:46.455Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces:         6,448 bytes
[task 2021-04-14T14:26:46.455Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Unused stack traces:       6,864 bytes
[task 2021-04-14T14:26:46.455Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Stack trace table:       196,640 bytes (16,384 entries, 64 used)
[task 2021-04-14T14:26:46.456Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Live block table:          4,128 bytes (64 entries, 30 used)
[task 2021-04-14T14:26:46.456Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Dead block table:             32 bytes (0 entries, 0 used)
[task 2021-04-14T14:26:46.456Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.456Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that are destroyed after Dump() ends {
[task 2021-04-14T14:26:46.456Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:         202,888 bytes
[task 2021-04-14T14:26:46.457Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces set:     12,288 bytes
[task 2021-04-14T14:26:46.457Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used PCs set:              12,288 bytes
[task 2021-04-14T14:26:46.457Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Pointer ID map:            20,480 bytes
[task 2021-04-14T14:26:46.457Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.457Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Counts {
[task 2021-04-14T14:26:46.458Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:            40 requests
[task 2021-04-14T14:26:46.460Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service cache:   0.0% hit rate, 1.0% occupancy at end
[task 2021-04-14T14:26:46.460Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.460Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   }
[task 2021-04-14T14:26:46.460Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.461Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Unknown pointer 0xa
[task 2021-04-14T14:26:46.461Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 6 {
[task 2021-04-14T14:26:46.461Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.461Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.461Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.462Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Execution measurements {
[task 2021-04-14T14:26:46.462Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that persist after Dump() ends {
[task 2021-04-14T14:26:46.462Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces:         3,536 bytes
[task 2021-04-14T14:26:46.462Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Unused stack traces:      19,344 bytes
[task 2021-04-14T14:26:46.463Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Stack trace table:       196,640 bytes (16,384 entries, 110 used)
[task 2021-04-14T14:26:46.463Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Live block table:          4,128 bytes (64 entries, 27 used)
[task 2021-04-14T14:26:46.463Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Dead block table:             32 bytes (0 entries, 0 used)
[task 2021-04-14T14:26:46.464Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.464Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that are destroyed after Dump() ends {
[task 2021-04-14T14:26:46.464Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:         202,888 bytes
[task 2021-04-14T14:26:46.465Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces set:     12,288 bytes
[task 2021-04-14T14:26:46.465Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used PCs set:              12,288 bytes
[task 2021-04-14T14:26:46.465Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Pointer ID map:            20,480 bytes
[task 2021-04-14T14:26:46.465Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.466Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Counts {
[task 2021-04-14T14:26:46.466Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:            27 requests
[task 2021-04-14T14:26:46.466Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service cache:   0.0% hit rate, 0.7% occupancy at end
[task 2021-04-14T14:26:46.467Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.467Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   }
[task 2021-04-14T14:26:46.467Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.469Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 7 {
[task 2021-04-14T14:26:46.470Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.470Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.470Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.470Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Execution measurements {
[task 2021-04-14T14:26:46.471Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that persist after Dump() ends {
[task 2021-04-14T14:26:46.471Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces:         3,536 bytes
[task 2021-04-14T14:26:46.471Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Unused stack traces:      23,296 bytes
[task 2021-04-14T14:26:46.471Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Stack trace table:       196,640 bytes (16,384 entries, 129 used)
[task 2021-04-14T14:26:46.472Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Live block table:          4,128 bytes (64 entries, 27 used)
[task 2021-04-14T14:26:46.472Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Dead block table:          1,056 bytes (16 entries, 9 used)
[task 2021-04-14T14:26:46.472Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.472Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Data structures that are destroyed after Dump() ends {
[task 2021-04-14T14:26:46.473Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:         202,888 bytes
[task 2021-04-14T14:26:46.473Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used stack traces set:     12,288 bytes
[task 2021-04-14T14:26:46.473Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Used PCs set:              12,288 bytes
[task 2021-04-14T14:26:46.474Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Pointer ID map:            20,480 bytes
[task 2021-04-14T14:26:46.474Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.474Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     Counts {
[task 2021-04-14T14:26:46.475Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service:            27 requests
[task 2021-04-14T14:26:46.475Z] 14:26:46     INFO -  PID 9325 | DMD[9337]       Location service cache:   0.0% hit rate, 0.7% occupancy at end
[task 2021-04-14T14:26:46.483Z] 14:26:46     INFO -  PID 9325 | DMD[9337]     }
[task 2021-04-14T14:26:46.483Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   }
[task 2021-04-14T14:26:46.484Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.485Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 8 {
[task 2021-04-14T14:26:46.486Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.486Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.487Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.487Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.488Z] 14:26:46     INFO -  PID 9325 | DMD[9337] Dump 9 {
[task 2021-04-14T14:26:46.489Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the heap block list...
[task 2021-04-14T14:26:46.490Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack trace table...
[task 2021-04-14T14:26:46.492Z] 14:26:46     INFO -  PID 9325 | DMD[9337]   Constructing the stack frame table...
[task 2021-04-14T14:26:46.493Z] 14:26:46     INFO -  PID 9325 | DMD[9337] }
[task 2021-04-14T14:26:46.493Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-empty-live - true == true
[task 2021-04-14T14:26:46.493Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-empty-dark-matter - true == true
[task 2021-04-14T14:26:46.494Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-empty-cumulative - true == true
[task 2021-04-14T14:26:46.495Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-full1-live - true == true
[task 2021-04-14T14:26:46.495Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-full1-dark-matter - true == true
[task 2021-04-14T14:26:46.496Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-full2-dark-matter - true == true
[task 2021-04-14T14:26:46.497Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-full2-cumulative - true == true
[task 2021-04-14T14:26:46.498Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] complete-partial-live - true == true
[task 2021-04-14T14:26:46.498Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | run_test - [run_test : 168] Basic scan test - true == true
[task 2021-04-14T14:26:46.499Z] 14:26:46     INFO -  PID 9325 | Creating block range list.
[task 2021-04-14T14:26:46.500Z] 14:26:46     INFO -  PID 9325 | Clamping block contents.
[task 2021-04-14T14:26:46.500Z] 14:26:46     INFO -  PID 9325 | Saving file.
[task 2021-04-14T14:26:46.501Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] basic-scan-64 - true == true
[task 2021-04-14T14:26:46.502Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | run_test - [run_test : 173] Scan with address clamping - true == true
[task 2021-04-14T14:26:46.503Z] 14:26:46     INFO -  PID 9325 | Creating block range list.
[task 2021-04-14T14:26:46.503Z] 14:26:46     INFO -  PID 9325 | Clamping block contents.
[task 2021-04-14T14:26:46.504Z] 14:26:46     INFO -  PID 9325 | Saving file.
[task 2021-04-14T14:26:46.505Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] basic-scan-64 - true == true
[task 2021-04-14T14:26:46.505Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] script-max-frames-8 - true == true
[task 2021-04-14T14:26:46.506Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] script-max-frames-3 - true == true
[task 2021-04-14T14:26:46.507Z] 14:26:46     INFO -  TEST-PASS | memory/replace/dmd/test/test_dmd.js | test - [test : 118] script-max-frames-1 - true == true
[task 2021-04-14T14:26:46.508Z] 14:26:46     INFO -  PID 9325 | Traceback (most recent call last):
[task 2021-04-14T14:26:46.508Z] 14:26:46     INFO -  PID 9325 |   File "/builds/worker/workspace/build/application/firefox/dmd.py", line 1010, in <module>
[task 2021-04-14T14:26:46.509Z] 14:26:46     INFO -  PID 9325 |     main()
[task 2021-04-14T14:26:46.510Z] 14:26:46     INFO -  PID 9325 |   File "/builds/worker/workspace/build/application/firefox/dmd.py", line 1002, in main
[task 2021-04-14T14:26:46.510Z] 14:26:46     INFO -  PID 9325 |     digest = getDigestFromFile(args, args.input_file)
[task 2021-04-14T14:26:46.511Z] 14:26:46     INFO -  PID 9325 |   File "/builds/worker/workspace/build/application/firefox/dmd.py", line 313, in getDigestFromFile
[task 2021-04-14T14:26:46.512Z] 14:26:46     INFO -  PID 9325 |     j = json.load(f)
[task 2021-04-14T14:26:46.513Z] 14:26:46     INFO -  PID 9325 |   File "/usr/lib/python3.6/json/__init__.py", line 299, in load
[task 2021-04-14T14:26:46.513Z] 14:26:46     INFO -  PID 9325 |     parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
[task 2021-04-14T14:26:46.514Z] 14:26:46     INFO -  PID 9325 |   File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
[task 2021-04-14T14:26:46.515Z] 14:26:46     INFO -  PID 9325 |     return _default_decoder.decode(s)
[task 2021-04-14T14:26:46.515Z] 14:26:46     INFO -  PID 9325 |   File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
[task 2021-04-14T14:26:46.516Z] 14:26:46     INFO -  PID 9325 |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
[task 2021-04-14T14:26:46.516Z] 14:26:46     INFO -  PID 9325 |   File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
[task 2021-04-14T14:26:46.517Z] 14:26:46     INFO -  PID 9325 |     raise JSONDecodeError("Expecting value", s, err.value) from None
[task 2021-04-14T14:26:46.518Z] 14:26:46     INFO -  PID 9325 | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[task 2021-04-14T14:26:46.518Z] 14:26:46     INFO -  PID 9325 | --- /builds/worker/workspace/build/tests/xpcshell/tests/memory/replace/dmd/test/script-sort-by-usable-expected.txt	2016-01-01 00:00:00.000000000 +0000
[task 2021-04-14T14:26:46.520Z] 14:26:46     INFO -  PID 9325 | +++ /builds/worker/workspace/build/tests/xpcshell/tests/memory/replace/dmd/test/script-sort-by-usable-actual.txt	2021-04-14 14:26:46.408489000 +0000
[task 2021-04-14T14:26:46.521Z] 14:26:46     INFO -  PID 9325 | @@ -1,46 +0,0 @@
[task 2021-04-14T14:26:46.522Z] 14:26:46     INFO -  PID 9325 | -#-----------------------------------------------------------------
[task 2021-04-14T14:26:46.522Z] 14:26:46     INFO -  PID 9325 | -# dmd.py --filter-stacks-for-testing -o script-sort-by-usable-actual.txt --sort-by=usable script-sort-by.json.gz
[task 2021-04-14T14:26:46.523Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.524Z] 14:26:46     INFO -  PID 9325 | -Invocation {
[task 2021-04-14T14:26:46.524Z] 14:26:46     INFO -  PID 9325 | -  $DMD = '--mode=live'
[task 2021-04-14T14:26:46.525Z] 14:26:46     INFO -  PID 9325 | -  Mode = 'live'
[task 2021-04-14T14:26:46.526Z] 14:26:46     INFO -  PID 9325 | -}
[task 2021-04-14T14:26:46.526Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.527Z] 14:26:46     INFO -  PID 9325 | -#-----------------------------------------------------------------
[task 2021-04-14T14:26:46.528Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.528Z] 14:26:46     INFO -  PID 9325 | -Live {
[task 2021-04-14T14:26:46.529Z] 14:26:46     INFO -  PID 9325 | -  5 blocks in heap block record 1 of 3
[task 2021-04-14T14:26:46.530Z] 14:26:46     INFO -  PID 9325 | -  16,400 bytes (12,016 requested / 4,384 slop)
[task 2021-04-14T14:26:46.530Z] 14:26:46     INFO -  PID 9325 | -  Individual block sizes: 4,096 x 4; 16
[task 2021-04-14T14:26:46.531Z] 14:26:46     INFO -  PID 9325 | -  33.35% of the heap (33.35% cumulative)
[task 2021-04-14T14:26:46.532Z] 14:26:46     INFO -  PID 9325 | -  Allocated at {
[task 2021-04-14T14:26:46.532Z] 14:26:46     INFO -  PID 9325 | -    #01: B (B.cpp:99)
[task 2021-04-14T14:26:46.533Z] 14:26:46     INFO -  PID 9325 | -  }
[task 2021-04-14T14:26:46.534Z] 14:26:46     INFO -  PID 9325 | -}
[task 2021-04-14T14:26:46.534Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.535Z] 14:26:46     INFO -  PID 9325 | -Live {
[task 2021-04-14T14:26:46.536Z] 14:26:46     INFO -  PID 9325 | -  5 blocks in heap block record 2 of 3
[task 2021-04-14T14:26:46.536Z] 14:26:46     INFO -  PID 9325 | -  16,392 bytes (16,392 requested / 0 slop)
[task 2021-04-14T14:26:46.537Z] 14:26:46     INFO -  PID 9325 | -  Individual block sizes: 4,096 x 4; 8
[task 2021-04-14T14:26:46.538Z] 14:26:46     INFO -  PID 9325 | -  33.33% of the heap (66.68% cumulative)
[task 2021-04-14T14:26:46.538Z] 14:26:46     INFO -  PID 9325 | -  Allocated at {
[task 2021-04-14T14:26:46.539Z] 14:26:46     INFO -  PID 9325 | -    #01: A (A.cpp:99)
[task 2021-04-14T14:26:46.540Z] 14:26:46     INFO -  PID 9325 | -  }
[task 2021-04-14T14:26:46.540Z] 14:26:46     INFO -  PID 9325 | -}
[task 2021-04-14T14:26:46.541Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.542Z] 14:26:46     INFO -  PID 9325 | -Live {
[task 2021-04-14T14:26:46.542Z] 14:26:46     INFO -  PID 9325 | -  8 blocks in heap block record 3 of 3
[task 2021-04-14T14:26:46.543Z] 14:26:46     INFO -  PID 9325 | -  16,384 bytes (8,200 requested / 8,184 slop)
[task 2021-04-14T14:26:46.544Z] 14:26:46     INFO -  PID 9325 | -  Individual block sizes: 2,048 x 8
[task 2021-04-14T14:26:46.544Z] 14:26:46     INFO -  PID 9325 | -  33.32% of the heap (100.00% cumulative)
[task 2021-04-14T14:26:46.545Z] 14:26:46     INFO -  PID 9325 | -  Allocated at {
[task 2021-04-14T14:26:46.546Z] 14:26:46     INFO -  PID 9325 | -    #01: C (C.cpp:99)
[task 2021-04-14T14:26:46.546Z] 14:26:46     INFO -  PID 9325 | -  }
[task 2021-04-14T14:26:46.547Z] 14:26:46     INFO -  PID 9325 | -}
[task 2021-04-14T14:26:46.548Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.548Z] 14:26:46     INFO -  PID 9325 | -#-----------------------------------------------------------------
[task 2021-04-14T14:26:46.549Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.550Z] 14:26:46     INFO -  PID 9325 | -Summary {
[task 2021-04-14T14:26:46.551Z] 14:26:46     INFO -  PID 9325 | -  Total: 49,176 bytes in 18 blocks
[task 2021-04-14T14:26:46.551Z] 14:26:46     INFO -  PID 9325 | -}
[task 2021-04-14T14:26:46.552Z] 14:26:46     INFO -  PID 9325 | -
[task 2021-04-14T14:26:46.553Z] 14:26:46  WARNING -  TEST-UNEXPECTED-FAIL | memory/replace/dmd/test/test_dmd.js | test - [test : 118] script-sort-by-usable - false == true
[task 2021-04-14T14:26:46.553Z] 14:26:46     INFO -  /builds/worker/workspace/build/tests/xpcshell/tests/memory/replace/dmd/test/test_dmd.js:test:118
[task 2021-04-14T14:26:46.554Z] 14:26:46     INFO -  /builds/worker/workspace/build/tests/xpcshell/tests/memory/replace/dmd/test/test_dmd.js:run_test:205
[task 2021-04-14T14:26:46.555Z] 14:26:46     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:_execute_test:592
[task 2021-04-14T14:26:46.556Z] 14:26:46     INFO -  -e:null:1
[task 2021-04-14T14:26:46.556Z] 14:26:46     INFO -  exiting test
[task 2021-04-14T14:26:46.557Z] 14:26:46     INFO -  <<<<<<<
[task 2021-04-14T14:26:46.558Z] 14:26:46     INFO -  TEST-START | toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js
[task 2021-04-14T14:26:56.751Z] 14:26:56     INFO -  TEST-PASS | toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | took 10289ms
[task 2021-04-14T14:26:56.757Z] 14:26:56     INFO -  INFO | Result summary:
[task 2021-04-14T14:26:56.758Z] 14:26:56     INFO -  INFO | Passed: 261
[task 2021-04-14T14:26:56.758Z] 14:26:56  WARNING -  INFO | Failed: 1
[task 2021-04-14T14:26:56.758Z] 14:26:56  WARNING -  One or more unittests failed.
[task 2021-04-14T14:26:56.758Z] 14:26:56     INFO -  INFO | Todo: 0
[task 2021-04-14T14:26:56.758Z] 14:26:56     INFO -  INFO | Retried: 2
[task 2021-04-14T14:26:56.758Z] 14:26:56     INFO -  SUITE-END | took 114s
[task 2021-04-14T14:26:56.758Z] 14:26:56     INFO -  Node moz-http2 server shutting down ...
[task 2021-04-14T14:26:56.758Z] 14:26:56     INFO -  http3Server server shutting down ...
[task 2021-04-14T14:26:56.899Z] 14:26:56    ERROR - Return code: 1
[task 2021-04-14T14:26:56.899Z] 14:26:56     INFO - TinderboxPrint: xpcshell-xpcshell<br/>261/<em class="testfail">1</em>/0

https://treeherder.mozilla.org/logviewer?job_id=336505292&repo=autoland&lineNumber=1941

[task 2021-04-14T14:31:00.938Z] 14:31:00     INFO -  TEST-START | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js
[task 2021-04-14T14:31:01.487Z] 14:31:01  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | xpcshell return code: 0
[task 2021-04-14T14:31:01.488Z] 14:31:01     INFO -  TEST-INFO took 552ms
[task 2021-04-14T14:31:01.488Z] 14:31:01     INFO -  >>>>>>>
[task 2021-04-14T14:31:01.488Z] 14:31:01     INFO -  PID 7752 | [Parent 7752, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:01.489Z] 14:31:01     INFO -  PID 7752 | [Parent 7752, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:01.489Z] 14:31:01     INFO -  PID 7752 | [Parent 7752, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:01.490Z] 14:31:01     INFO -  PID 7752 | [Parent 7752, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:01.490Z] 14:31:01     INFO -  PID 7752 | [Parent 7752, Main Thread] WARNING: Couldn't get the user appdata directory. Crash events may not be produced.: file /builds/worker/checkouts/gecko/toolkit/crashreporter/nsExceptionHandler.cpp:2942
[task 2021-04-14T14:31:01.490Z] 14:31:01     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2021-04-14T14:31:01.491Z] 14:31:01     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2021-04-14T14:31:01.491Z] 14:31:01     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2021-04-14T14:31:01.492Z] 14:31:01     INFO -  running event loop
[task 2021-04-14T14:31:01.492Z] 14:31:01     INFO -  toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | Starting setup
[task 2021-04-14T14:31:01.492Z] 14:31:01     INFO -  (xpcshell/head.js) | test setup pending (2)
[task 2021-04-14T14:31:01.493Z] 14:31:01     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2021-04-14T14:31:01.493Z] 14:31:01     INFO -  PID 7752 | [Parent 7752, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, kKnownEsrVersion) failed with result 0x80004002 (NS_NOINTERFACE): file /builds/worker/checkouts/gecko/toolkit/components/resistfingerprinting/nsRFPService.cpp:553
[task 2021-04-14T14:31:01.493Z] 14:31:01     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
[task 2021-04-14T14:31:01.493Z] 14:31:01     INFO -  (xpcshell/head.js) | test setup finished (2)
[task 2021-04-14T14:31:01.493Z] 14:31:01     INFO -  toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | Starting test_subprocess_io
[task 2021-04-14T14:31:01.494Z] 14:31:01     INFO -  (xpcshell/head.js) | test test_subprocess_io pending (2)
[task 2021-04-14T14:31:01.494Z] 14:31:01     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
[task 2021-04-14T14:31:01.494Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 51] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 54] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 58] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 61] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 65] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 68] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 72] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 75] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 78] ..
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 | Traceback (most recent call last):
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 |   File "Z:\\task_1618409729\\build\\tests\\xpcshell\\tests\\toolkit\\modules\\subprocess\\test\\xpcshell\\data_test_script.py", line 35, in <module>
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 |     echo_loop()
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 |   File "Z:\\task_1618409729\\build\\tests\\xpcshell\\tests\\toolkit\\modules\\subprocess\\test\\xpcshell\\data_test_script.py", line 20, in echo_loop
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 |     line = sys.stdin.buffer.readline()
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 | AttributeError: 'file' object has no attribute 'buffer'
[task 2021-04-14T14:31:01.495Z] 14:31:01     INFO -  PID 7752 | JavaScript error: Z:\\task_1618409729\\build\\tests\\xpcshell\\head.js, line 241: uncaught exception: Object
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  "CONSOLE_MESSAGE: (error) [JavaScript Error: "uncaught exception: Object" {file: "Z:\\task_1618409729\\build\\tests\\xpcshell\\head.js" line: 241}]"
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  Unexpected exception File closed at resource://gre/modules/subprocess/subprocess_worker_win.js:89
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  close@resource://gre/modules/subprocess/subprocess_worker_win.js:89:19
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  onError@resource://gre/modules/subprocess/subprocess_worker_win.js:117:10
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  onReady@resource://gre/modules/subprocess/subprocess_worker_win.js:217:12
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  poll@resource://gre/modules/subprocess/subprocess_worker_win.js:768:28
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  loop@resource://gre/modules/subprocess/subprocess_worker_win.js:747:10
[task 2021-04-14T14:31:01.496Z] 14:31:01     INFO -  exiting test
[task 2021-04-14T14:31:02.071Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: A runnable was posted to a worker that is already shutting down!: file /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:1471
[task 2021-04-14T14:31:02.071Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: Failed to dispatch offline status change event!: file /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:1956
[task 2021-04-14T14:31:02.071Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: A runnable was posted to a worker that is already shutting down!: file /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:1471
[task 2021-04-14T14:31:02.071Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: Failed to dispatch offline status change event!: file /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:1956
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3361
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: '!obs', file /builds/worker/checkouts/gecko/toolkit/components/sessionstore/RestoreTabContentObserver.cpp:58
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: OOPDeinit() without successful OOPInit(): file /builds/worker/checkouts/gecko/toolkit/crashreporter/nsExceptionHandler.cpp:3398
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4442
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/checkouts/gecko/xpcom/base/nsTraceRefcnt.cpp:202
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  PID 3096 | [Parent 3096, Main Thread] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/checkouts/gecko/xpcom/base/nsTraceRefcnt.cpp:202
[task 2021-04-14T14:31:02.072Z] 14:31:02     INFO -  <<<<<<<
[task 2021-04-14T14:31:02.077Z] 14:31:02     INFO -  TEST-START | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js
[task 2021-04-14T14:31:02.643Z] 14:31:02  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | xpcshell return code: 0
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  TEST-INFO took 560ms
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  >>>>>>>
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  PID 10388 | [Parent 10388, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  PID 10388 | [Parent 10388, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  PID 10388 | [Parent 10388, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  PID 10388 | [Parent 10388, Main Thread] WARNING: Failed to get directory to cache.: file /builds/worker/checkouts/gecko/security/sandbox/win/src/sandboxbroker/sandboxBroker.cpp:119
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  PID 10388 | [Parent 10388, Main Thread] WARNING: Couldn't get the user appdata directory. Crash events may not be produced.: file /builds/worker/checkouts/gecko/toolkit/crashreporter/nsExceptionHandler.cpp:2942
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2021-04-14T14:31:02.643Z] 14:31:02     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  running event loop
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | Starting setup
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test setup pending (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  PID 10388 | [Parent 10388, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, kKnownEsrVersion) failed with result 0x80004002 (NS_NOINTERFACE): file /builds/worker/checkouts/gecko/toolkit/components/resistfingerprinting/nsRFPService.cpp:553
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test setup finished (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | Starting test_subprocess_io
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test test_subprocess_io pending (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 51] ..
[task 2021-04-14T14:31:02.644Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 54] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 58] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 61] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 65] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 68] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 72] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 75] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  TEST-PASS | toolkit/modules/subprocess/test/xpcshell/test_subprocess.js | test_subprocess_io - [test_subprocess_io : 78] ..
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 | Traceback (most recent call last):
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 |   File "Z:\\task_1618409729\\build\\tests\\xpcshell\\tests\\toolkit\\modules\\subprocess\\test\\xpcshell\\data_test_script.py", line 35, in <module>
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 |     echo_loop()
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 |   File "Z:\\task_1618409729\\build\\tests\\xpcshell\\tests\\toolkit\\modules\\subprocess\\test\\xpcshell\\data_test_script.py", line 20, in echo_loop
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 |     line = sys.stdin.buffer.readline()
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 | AttributeError: 'file' object has no attribute 'buffer'
[task 2021-04-14T14:31:02.645Z] 14:31:02     INFO -  PID 10388 | JavaScript error: Z:\\task_1618409729\\build\\tests\\xpcshell\\head.js, line 241: uncaught exception: Object
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  "CONSOLE_MESSAGE: (error) [JavaScript Error: "uncaught exception: Object" {file: "Z:\\task_1618409729\\build\\tests\\xpcshell\\head.js" line: 241}]"
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  Unexpected exception File closed at resource://gre/modules/subprocess/subprocess_worker_win.js:89
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  close@resource://gre/modules/subprocess/subprocess_worker_win.js:89:19
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  onError@resource://gre/modules/subprocess/subprocess_worker_win.js:117:10
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  onReady@resource://gre/modules/subprocess/subprocess_worker_win.js:217:12
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  poll@resource://gre/modules/subprocess/subprocess_worker_win.js:768:28
[task 2021-04-14T14:31:02.646Z] 14:31:02     INFO -  loop@resource://gre/modules/subprocess/subprocess_worker_win.js:747:10
...
...
...
Flags: needinfo?(bhearsum)

I'm pretty sure ahal is taking care of this.

Flags: needinfo?(bhearsum)

Sadly the TV failures in the backout are not pre-existing. Looking at a log, I see:

[task 2021-04-14T15:45:47.395Z] 15:45:47     INFO -  PID 5548 | Traceback (most recent call last):
[task 2021-04-14T15:45:47.398Z] 15:45:47     INFO -  PID 5548 |   File "/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/modules/subprocess/test/xpcshell/data_test_script.py", line 35, in <module>
[task 2021-04-14T15:45:47.399Z] 15:45:47     INFO -  PID 5548 |     echo_loop()
[task 2021-04-14T15:45:47.399Z] 15:45:47     INFO -  PID 5548 |   File "/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/modules/subprocess/test/xpcshell/data_test_script.py", line 20, in echo_loop
[task 2021-04-14T15:45:47.399Z] 15:45:47     INFO -  PID 5548 |     line = sys.stdin.buffer.readline()
[task 2021-04-14T15:45:47.399Z] 15:45:47     INFO -  PID 5548 | AttributeError: 'file' object has no attribute 'buffer'

So something about the context TV is running with is causing our fix in data_test_script.py to fail.

Attachment #9211506 - Attachment description: WIP: Bug 1696531: use python3 for xpcshell tests → Bug 1696531: use python3 for xpcshell tests, r?bhearsum!
Attachment #9220471 - Attachment description: WIP: Bug 1696531 - [xpcshell] Hack around Python bug on Windows with Python < 3.7.5 → Bug 1696531 - [xpcshell] Hack around Python bug on Windows with Python < 3.7.5, r?#releng-reviewers!
Attachment #9215139 - Attachment description: Bug 1696531 - Use a shorter name in 'test_on_directory' from test_unix_domain.js, r?bhearsum → Bug 1696531 - Use a shorter name in 'test_on_directory' from test_unix_domain.js, r?bhearsum!
Attachment #9215272 - Attachment description: Bug 1696531 - Keep Python 2 compatibility in tools/rb/fix_stacks.py for now, r?bhearsum → Bug 1696531 - Keep Python 2 compatibility in tools/rb/fix_stacks.py for now, r?bhearsum!
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e5e76f56ceb9
fix fake adb process to support python2 and 3 r=ahal
https://hg.mozilla.org/integration/autoland/rev/ddcc795bf838
fix dmd.py and related tools to support python 3 r=ahal
https://hg.mozilla.org/integration/autoland/rev/544be24f74be
add required import to xpcshell/selftest.py r=ahal
https://hg.mozilla.org/integration/autoland/rev/b0eee4af2caf
[xpcshell] Fix Python 3 error in selftest.py, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/98d17a5f6886
[xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/9557ff3065bc
Use a shorter name in 'test_on_directory' from test_unix_domain.js, r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/c3f229148f6c
Keep Python 2 compatibility in tools/rb/fix_stacks.py for now, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/97771570e425
[xpcshell] Skip test_subprocess.js in test-verify mode on MacOs, r=releng-reviewers,jmaher
https://hg.mozilla.org/integration/autoland/rev/82445e26060e
[xpcshell] Hack around Python bug on Windows with Python < 3.7.5, r=releng-reviewers,jmaher
https://hg.mozilla.org/integration/autoland/rev/abc85e3c21b0
use python3 for xpcshell tests, r=bhearsum

Backed out for causing mochitest failures in TypeError: cannot use a string pattern on a bytes-like object

Backout link: https://hg.mozilla.org/integration/autoland/rev/1019c6353490755bc335b1bac9e662afcf7babe5

Push with failures

Failure log

Flags: needinfo?(ahal)
Flags: needinfo?(ahal)

I'm going to start landing this stack a few commits at a time to try and make it a bit more sane (and easier to tell what is causing these mochitest failures)

Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/90ee1608f59b
Use a shorter name in 'test_on_directory' from test_unix_domain.js, r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/3581ed96ea2a
[xpcshell] Skip test_subprocess.js in test-verify mode on MacOs, r=releng-reviewers,jmaher
https://hg.mozilla.org/integration/autoland/rev/bab03cad0edc
[xpcshell] Hack around Python bug on Windows with Python < 3.7.5, r=releng-reviewers,jmaher
https://hg.mozilla.org/integration/autoland/rev/68918f515310
add required import to xpcshell/selftest.py r=ahal
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2436af0e91c8
[xpcshell] Fix Python 3 error in selftest.py, r=bhearsum

That seems fairly unlikely. Alexandru, perhaps that was caused by something like bug 1707310?

Flags: needinfo?(aionescu)
Flags: needinfo?(aionescu)
Attachment #9211509 - Attachment description: Bug 1696531 - [xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r?bhearsum → Bug 1696531 - [xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r?bhearsum!
Attachment #9211507 - Attachment description: Bug 1696531: fix fake adb process to support python2 and 3 r=ahal → Bug 1696531: fix fake adb process to support python2 and 3, r?bhearsum!
Attachment #9211510 - Attachment description: Bug 1696531: fix dmd.py and related tools to support python 3 r=ahal → Bug 1696531: fix dmd.py and related tools to support python 3, r?bhearsum!

The 'subprocess.list2cmdline' function fails if you pass in a byte-string on
Python 3, and mozcrash is explicitly encoding that value, so I believe it will
always be an exception when this code path is hit.

I'm a bit surprised because mochitest also calls into this function and has been
running with Python 3 for months now. Since xpcshell and mochitest are the only
things using this and both will be on Python 3, we don't need to worry about
maintaining Python 2 compatibility here.

Depends on D109731

Attachment #9215272 - Attachment is obsolete: true
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f67fc384b729
[xpcshell] Fix some py3 related issues in toolkit/modules/subprocess/test/xpcshell/test_subprocess.js, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/a7b265c9fbc2
fix fake adb process to support python2 and 3, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/5cda942e114a
fix dmd.py and related tools to support python 3, r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/3ad44bb73874
Remove erroneous encoding of subprocess arg in 'mozcrash', r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/3bfb97d0825a
use python3 for xpcshell tests, r=bhearsum
See Also: → 1716370
Status: NEW → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.