1950855 - mach xpcshell-test fails on Android: shutil.Error in copytree after updating checkout due to dangling symlinks

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Rob Wu [:robwu]]

I updated my checkout, ran ./mach build and tried to run a test on an Android emulator, as follows:

./mach xpcshell-test toolkit/mozapps/extensions/test/xpcshell/test_shutdown_barriers.js --verbose

This fails (details below), and I believe that this is due to the dangling symlinks in the objdir, that have been created as part of an earlier build. While I could try clobbering the objdir and rebuild, I think that it would be better to just ignore such symlinks, if any.

The above results in the following error, probably due to the removal of the files in bug 1949027

shutil.Error: [('/path-to-gecko-objdir/_tests/xpcshell/security/manager/ssl/tests/unit/test_crlite_filter_corrupted.js', '/var/folders/xx/xxx/T/tmpgei1jma2/xpc/security/manager/ssl/tests/unit/test_crlite_filter_corrupted.js', "[Errno 2] No such file or directory: '/path-to-gecko-objdir/_tests/xpcshell/security/manager/ssl/tests/unit/test_crlite_filter_corrupted.js'") (...many more files...) , "No such file or directory: '/path-to-gecko-objdir/_tests/xpcshell/security/manager/ssl/tests/unit/test_builtins/ee-notBefore-2023.pem'")]

and the following stack trace

  File "/path/to/gecko/testing/xpcshell/mach_commands.py", line 314, in run_xpcshell_test
    return xpcshell.run_test(**params)
  File "/path/to/gecko/testing/xpcshell/mach_commands.py", line 186, in run_test
    xpcshell = remotexpcshelltests.XPCShellRemote(kwargs, log)
  File "/path/to/gecko/testing/xpcshell/remotexpcshelltests.py", line 459, in __init__
    self.setupTestDir()
  File "/path/to/gecko/testing/xpcshell/remotexpcshelltests.py", line 688, in setupTestDir
    self.device.push(self.xpcDir, self.remoteScriptsDir, timeout=600)
  File "/path/to/gecko/testing/mozbase/mozdevice/mozdevice/adb.py", line 3012, in push
    copytree(local, new_local)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/shutil.py", line 557, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/shutil.py", line 513, in _copytree
    raise Error(errors)

the copytree call happens at https://searchfox.org/mozilla-central/rev/80ae03d93e3fd5769b16f37719b610e359f8fc62/testing/mozbase/mozdevice/mozdevice/adb.py#3012

Comment 1

[Rob Wu [:robwu]]

I thought that it would be as trivial as passing ignore_dangling_symlinks=True to the shutil.copytree call, but that did not work for me because I am using Python 3.9, which does not apply the ignore_dangling_symlinks=True option recursively. The bug was fixed in Python 3.12 by https://github.com/python/cpython/commit/5ff81da6d3a8eb01fc5500fd1c9eaa6543286301. However, that implementation still has a bug: symlinks are resolved relative to the working dir and not the original location of the symlink: https://github.com/python/cpython/issues/91205#issuecomment-1866862469

To fix this, I can think of two approaches:

• catch the error and ignore files
• Switch to mozfile.copy_contents (and adjust the version check to expand to more Python versions, for now every Python version until https://github.com/python/cpython/issues/91205 is fixed). This requires mozdevice to have a new dependency, mozfile.

Yet another approach is to clear those dangling symlinks, e.g. from ./mach build. That would require the implementation to scan directories for unexpected files, which would slow down the build.

Comment 2

[Rob Wu [:robwu]]

Attachment: Bug 1950855 - Ignore dangling symlinks in adb.py

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/694c54e9e52c
Ignore dangling symlinks in adb.py r=ahal

Comment 4

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/694c54e9e52c