1713201 - Intermittent 5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_absolute_path TEST-UNEXPECTED-FAIL

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=340983010&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Ii8sLx2MQ7ygroioEPo_Fg/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-05-27T17:17:41.031Z]  5:23.21 Z:\task_1622133900\build\src\python\mozbuild\mozbuild\test\configure\test_toolchain_configure.py
[task 2021-05-27T17:17:41.031Z]  5:23.21 ============================= test session starts =============================
[task 2021-05-27T17:17:41.031Z]  5:23.21 platform win32 -- Python 3.6.5, pytest-3.6.2, py-1.10.0, pluggy-0.6.0 -- Z:\task_1622133900\build\src\obj-x86_64-pc-mingw32\_virtualenvs\python-test\Scripts\python.exe
[task 2021-05-27T17:17:41.032Z]  5:23.21 rootdir: Z:\task_1622133900\build\src, inifile: Z:\task_1622133900\build\src\config\mozunit\mozunit\pytest.ini
[task 2021-05-27T17:17:41.032Z]  5:23.21 collecting ... collected 54 items
[task 2021-05-27T17:17:41.032Z]  5:23.21 
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_absolute_path TEST-UNEXPECTED-FAIL
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_atypical_name PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_clang PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_default PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_gcc PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_guess_cxx PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_guess_cxx_clang PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_mismatched_compiler PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_mismatched_gcc PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_mixed_compilers PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_no_supported_compiler PASSED
[task 2021-05-27T17:17:41.032Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_overridden_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_unsupported_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_unsupported_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxSimpleCrossToolchainTest::test_cross_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxSimpleCrossToolchainTest::test_cross_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxX86_64CrossToolchainTest::test_cross_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxX86_64CrossToolchainTest::test_cross_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_forced_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_forced_unsupported_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_not_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_unsupported_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_clang_cl PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_overridden_supported_elsewhere_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_overridden_unsupported_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_overridden_unsupported_gcc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_unsupported_clang_cl PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_unsupported_msvc PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_clang PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_clang_cl PASSED
[task 2021-05-27T17:17:41.033Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_gcc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_overridden_supported_elsewhere_gcc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_overridden_unsupported_clang PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_overridden_unsupported_gcc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_unsupported_clang_cl PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::Windows64ToolchainTest::test_unsupported_msvc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cannot_cross PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cross_atypical_clang PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cross_clang PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cross_gcc_misc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cross_ppc_ppc64 PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cross_sparc_sparc64 PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_cross_x86_x64 PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_guess_cross_cxx PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_overridden_cross_gcc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxCrossCompileToolchainTest::test_overridden_unsupported_cross_gcc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXCrossToolchainTest::test_cannot_osx_cross PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXCrossToolchainTest::test_osx_cross PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsCrossToolchainTest::test_clang_cl_cross PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OpenBSDToolchainTest::test_gcc PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::RustTest::test_rust_target PASSED
[task 2021-05-27T17:17:41.034Z]  5:23.21 
[task 2021-05-27T17:17:41.034Z]  5:23.21 ================================== FAILURES ===================================
[task 2021-05-27T17:17:41.035Z]  5:23.21 ____________________ LinuxToolchainTest.test_absolute_path ____________________
[task 2021-05-27T17:17:41.035Z]  5:23.21 
[task 2021-05-27T17:17:41.035Z]  5:23.21 self = <test_toolchain_configure.LinuxToolchainTest testMethod=test_absolute_path>
[task 2021-05-27T17:17:41.035Z]  5:23.21 
[task 2021-05-27T17:17:41.035Z]  5:23.21     def test_absolute_path(self):
[task 2021-05-27T17:17:41.035Z]  5:23.21         paths = dict(self.PATHS)
[task 2021-05-27T17:17:41.035Z]  5:23.21         paths.update(
[task 2021-05-27T17:17:41.035Z]  5:23.21             {
[task 2021-05-27T17:17:41.035Z]  5:23.21                 "/opt/clang/bin/clang": paths["/usr/bin/clang"],
[task 2021-05-27T17:17:41.035Z]  5:23.21                 "/opt/clang/bin/clang++": paths["/usr/bin/clang++"],
[task 2021-05-27T17:17:41.035Z]  5:23.21             }
[task 2021-05-27T17:17:41.035Z]  5:23.21         )
[task 2021-05-27T17:17:41.035Z]  5:23.21         result = {
[task 2021-05-27T17:17:41.035Z]  5:23.21             "c_compiler": self.DEFAULT_CLANG_RESULT
[task 2021-05-27T17:17:41.035Z]  5:23.21             + {
[task 2021-05-27T17:17:41.035Z]  5:23.21                 "compiler": "/opt/clang/bin/clang",
[task 2021-05-27T17:17:41.035Z]  5:23.21             },
[task 2021-05-27T17:17:41.035Z]  5:23.21             "cxx_compiler": self.DEFAULT_CLANGXX_RESULT
[task 2021-05-27T17:17:41.035Z]  5:23.21             + {
[task 2021-05-27T17:17:41.035Z]  5:23.21                 "compiler": "/opt/clang/bin/clang++",
[task 2021-05-27T17:17:41.035Z]  5:23.21             },
[task 2021-05-27T17:17:41.035Z]  5:23.21         }
[task 2021-05-27T17:17:41.036Z]  5:23.21         self.do_toolchain_test(
[task 2021-05-27T17:17:41.036Z]  5:23.21             paths,
[task 2021-05-27T17:17:41.036Z]  5:23.21             result,
[task 2021-05-27T17:17:41.036Z]  5:23.21             environ={
[task 2021-05-27T17:17:41.036Z]  5:23.21                 "CC": "/opt/clang/bin/clang",
[task 2021-05-27T17:17:41.036Z]  5:23.21 >               "CXX": "/opt/clang/bin/clang++",
[task 2021-05-27T17:17:41.036Z]  5:23.21             },
[task 2021-05-27T17:17:41.036Z]  5:23.21         )
[task 2021-05-27T17:17:41.036Z]  5:23.21 
[task 2021-05-27T17:17:41.036Z]  5:23.21 python\mozbuild\mozbuild\test\configure\test_toolchain_configure.py:871:
[task 2021-05-27T17:17:41.036Z]  5:23.21 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2021-05-27T17:17:41.036Z]  5:23.21 python\mozbuild\mozbuild\test\configure\test_toolchain_configure.py:457: in do_toolchain_test
[task 2021-05-27T17:17:41.036Z]  5:23.21     sandbox = self.get_sandbox(paths, {}, args, environ, logger=self.logger)
[task 2021-05-27T17:17:41.036Z]  5:23.21 python\mozbuild\mozbuild\test\configure\common.py:318: in get_sandbox
[task 2021-05-27T17:17:41.036Z]  5:23.21     sandbox.include_file(os.path.join(topsrcdir, "moz.configure"))
[task 2021-05-27T17:17:41.036Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:493: in include_file
[task 2021-05-27T17:17:41.036Z]  5:23.21     exec_(code, self)
[task 2021-05-27T17:17:41.036Z]  5:23.21 python\mozbuild\mozbuild\util.py:55: in exec_
[task 2021-05-27T17:17:41.036Z]  5:23.21     exec(object, globals, locals)
[task 2021-05-27T17:17:41.036Z]  5:23.21 moz.configure:7: in <module>
[task 2021-05-27T17:17:41.036Z]  5:23.21     include("build/moz.configure/init.configure")
[task 2021-05-27T17:17:41.036Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:853: in include_impl
[task 2021-05-27T17:17:41.037Z]  5:23.21     self.include_file(what)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:493: in include_file
[task 2021-05-27T17:17:41.037Z]  5:23.21     exec_(code, self)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\util.py:55: in exec_
[task 2021-05-27T17:17:41.037Z]  5:23.21     exec(object, globals, locals)
[task 2021-05-27T17:17:41.037Z]  5:23.21 build\moz.configure\init.configure:233: in <module>
[task 2021-05-27T17:17:41.037Z]  5:23.21     @imports(_from="six", _import="ensure_text")
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:836: in decorator
[task 2021-05-27T17:17:41.037Z]  5:23.21     depends = DependsFunction(self, func, dependencies, when=when)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:137: in __init__
[task 2021-05-27T17:17:41.037Z]  5:23.21     sandbox._value_for(self)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:614: in _value_for
[task 2021-05-27T17:17:41.037Z]  5:23.21     return self._value_for_depends(obj)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\util.py:1049: in method_call
[task 2021-05-27T17:17:41.037Z]  5:23.21     cache[args] = self.func(instance, *args)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:623: in _value_for_depends
[task 2021-05-27T17:17:41.037Z]  5:23.21     value = obj.result()
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\util.py:1049: in method_call
[task 2021-05-27T17:17:41.037Z]  5:23.21     cache[args] = self.func(instance, *args)
[task 2021-05-27T17:17:41.037Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:162: in result
[task 2021-05-27T17:17:41.037Z]  5:23.21     return self._func(*resolved_args)
[task 2021-05-27T17:17:41.038Z]  5:23.21 python\mozbuild\mozbuild\configure\__init__.py:1298: in wrapped
[task 2021-05-27T17:17:41.038Z]  5:23.21     return new_func(*args, **kwargs)
[task 2021-05-27T17:17:41.038Z]  5:23.21 build\moz.configure\init.configure:338: in virtualenv_python3
[task 2021-05-27T17:17:41.038Z]  5:23.21     if not manager.up_to_date(python):
[task 2021-05-27T17:17:41.038Z]  5:23.21 python\mozbuild\mozbuild\virtualenv.py:203: in up_to_date
[task 2021-05-27T17:17:41.038Z]  5:23.21     orig_version = self.get_exe_info()

Comment 3

[Alexandru Michis [:malexandru]]

Mitchell, could this and Bug 1713353 be caused by one of your patches that landed here?

Comment 4

[Intermittent Failures Robot]

11 failures in 2908 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 10
• mozilla-central: 1

Platform and build breakdown:

• windows10-64: 11
  • opt: 11

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-05-24&endday=2021-05-30&tree=all

Comment 5

[Mitchell Hentges [:mhentges] 🦀]

Very possible, I'll investigate.

Comment 6

[Intermittent Failures Robot]

78 failures in 598 pushes (0.13 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 78

Platform and build breakdown:

• windows10-64: 77
  • opt: 77
• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-06-03&endday=2021-06-03&tree=all

Comment 7

[Mitchell Hentges [:mhentges] 🦀]

I've been pushing this week to get access to a quarantined Windows host, and I haven't had luck yet.
I'll continue the grind next week.

Comment 8

[Intermittent Failures Robot]

86 failures in 3065 pushes (0.028 failures/push) were associated with this bug in the last 7 days.

This is the #14 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 84
• mozilla-central: 2

Platform and build breakdown:

• windows10-64: 85
  • opt: 85
• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-05-31&endday=2021-06-06&tree=all

Comment 10

[Intermittent Failures Robot]

36 failures in 3502 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #49 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 29
• try: 1
• mozilla-central: 6

Platform and build breakdown:

• windows10-64: 36
  • opt: 36

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-06-07&endday=2021-06-13&tree=all

Comment 11

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1713201: Resolve test_telemetry failing locally

On Windows, Python's temporary directories are placed in %APPDATA%.
Unfortunately, the failing tests want to assume that some "other"
path (as a subpath of a tempdir) won't be within any
filter_args() known top-level paths, which includes the user's home
dir.

The only reason that this assumption has passed in CI so far is due
to Windows case insensitivity: running locally had ~ evaluated
to c:\Users\..., while running in CI had it be c:\users\....
Since the tempdir path started with C:\Users, filter_args()
would <path omitted> in CI but would $HOME/... locally.

By not basing the other_path on tempdir, we now consistently get the
same results across platforms.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed02a0713171
Resolve test_telemetry failing locally r=ahal

Comment 13

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ed02a0713171

Comment 14

[Intermittent Failures Robot]

40 failures in 574 pushes (0.07 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 38
• mozilla-central: 2

Platform and build breakdown:

• windows10-64: 40
  • opt: 40

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-06-16&endday=2021-06-16&tree=all

Comment 15

[Andreea Pavel [:apavel]]

Hi Mitchell, this still occurs after the fix landed in central, can you please take a look?

https://treeherder.mozilla.org/jobs?repo=mozilla-central&revision=1fb3d75a2240e9f22bb082de76620ce4919e8efd&selectedTaskRun=PsU30AIQScCPBCL8upldrg.0&searchStr=Windows%2C10%2Cx64%2Copt%2CPython%2C3%2Cunit%2Ctests%2Csource-test-python-mozbuild-windows10-64%2Fopt-py3%2Cmbu

Failure log: https://treeherder.mozilla.org/logviewer?job_id=343145669&repo=mozilla-central&lineNumber=2229

[task 2021-06-17T22:16:37.067Z] 1:15.14 Z:\task_1623967306\build\src\python\mozbuild\mozbuild\test\configure\test_moz_configure.py
[task 2021-06-17T22:16:37.067Z] 1:15.14 ============================= test session starts =============================
[task 2021-06-17T22:16:37.067Z] 1:15.14 platform win32 -- Python 3.6.5, pytest-3.6.2, py-1.10.0, pluggy-0.6.0 -- Z:\task_1623967306\build\src\obj-x86_64-pc-mingw32_virtualenvs\python-test\Scripts\python.exe
[task 2021-06-17T22:16:37.067Z] 1:15.14 rootdir: Z:\task_1623967306\build\src, inifile: Z:\task_1623967306\build\src\config\mozunit\mozunit\pytest.ini
[task 2021-06-17T22:16:37.068Z] 1:15.14 collecting ... collected 5 items
[task 2021-06-17T22:16:37.068Z] 1:15.14
[task 2021-06-17T22:16:37.068Z] 1:15.14 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetLinux::test_target PASSED
[task 2021-06-17T22:16:37.068Z] 1:15.14 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetWindows::test_target PASSED
[task 2021-06-17T22:16:37.068Z] 1:15.14 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetAndroid::test_target PASSED
[task 2021-06-17T22:16:37.068Z] 1:15.14 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetOpenBSD::test_target PASSED
[task 2021-06-17T22:16:37.068Z] 1:15.14 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestMozConfigure::test_nsis_version PASSED
[task 2021-06-17T22:16:37.068Z] 1:15.14
[task 2021-06-17T22:16:37.068Z] 1:15.14 ========================== 5 passed in 58.80 seconds ==========================
[task 2021-06-17T22:20:59.906Z] 5:37.98 Z:\task_1623967306\build\src\python\mozbuild\mozbuild\test\configure\test_toolchain_configure.py
[task 2021-06-17T22:20:59.906Z] 5:37.98 ============================= test session starts =============================
[task 2021-06-17T22:20:59.906Z] 5:37.98 platform win32 -- Python 3.6.5, pytest-3.6.2, py-1.10.0, pluggy-0.6.0 -- Z:\task_1623967306\build\src\obj-x86_64-pc-mingw32_virtualenvs\python-test\Scripts\python.exe
[task 2021-06-17T22:20:59.906Z] 5:37.98 rootdir: Z:\task_1623967306\build\src, inifile: Z:\task_1623967306\build\src\config\mozunit\mozunit\pytest.ini
[task 2021-06-17T22:20:59.906Z] 5:37.98 collecting ... collected 54 items
[task 2021-06-17T22:20:59.906Z] 5:37.98
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_absolute_path TEST-UNEXPECTED-FAIL
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_atypical_name PASSED
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_clang PASSED
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_default PASSED
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_gcc PASSED
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_guess_cxx PASSED
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_guess_cxx_clang PASSED
[task 2021-06-17T22:20:59.906Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_mismatched_compiler PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_mismatched_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_mixed_compilers PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_no_supported_compiler PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_overridden_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_unsupported_clang PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxToolchainTest::test_unsupported_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxSimpleCrossToolchainTest::test_cross_clang PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxSimpleCrossToolchainTest::test_cross_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxX86_64CrossToolchainTest::test_cross_clang PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::LinuxX86_64CrossToolchainTest::test_cross_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_clang PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_forced_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_forced_unsupported_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_not_gcc PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXToolchainTest::test_unsupported_clang PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_clang PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_clang_cl PASSED
[task 2021-06-17T22:20:59.907Z] 5:37.98 python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::WindowsToolchainTest::test_gcc PASSED

Comment 16

[Intermittent Failures Robot]

76 failures in 2888 pushes (0.026 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-beta: 2
• autoland: 68
• mozilla-central: 6

Platform and build breakdown:

• windows10-64: 76
  • opt: 76

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-06-14&endday=2021-06-20&tree=all

Comment 17

[Mitchell Hentges [:mhentges] 🦀]

Yeah, that preview patch addressed a related part of the problem, but I'm still working towards the fix.
I've got a reproduce on RDP'd TC workers, but:

• They get recycled after a few hours, killing my connection with them
• It takes ~20m for each iterative test
• It's intermittent >:(

Still working on this, I'll mark as FIXED/remove leave-open once it appears fixed from my end :)

Comment 18

[Intermittent Failures Robot]

19 failures in 3115 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 17
• mozilla-central: 2

Platform and build breakdown:

• windows10-64: 19
  • opt: 19

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-06-21&endday=2021-06-27&tree=all

Comment 19

[Mitchell Hentges [:mhentges] 🦀]

This may be resolved by 1718156 due to the common virtualenv no longer being created.

Comment 20

[Mitchell Hentges [:mhentges] 🦀]

This doesn't appear to be happening anymore now that we aren't creating virtualenvs in the tests.

It'd be interested to see why the test was failing intermittently on Windows, but there's work to be done that has a better reward-to-time payoff.
Could be interesting to dig into in the future.

Comment 21

[Mike Hommey [:glandium]]

why the test was failing intermittently on Windows

presumably because Windows doesn't allow some operations on open files, combined with some sort of race condition from multiple tests running in parallel.

Comment 22

[Intermittent Failures Robot]

4 failures in 2387 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 3

Platform and build breakdown:

• windows10-64: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1713201&startday=2021-07-05&endday=2021-07-11&tree=all