Closed Bug 1755569 Opened 3 years ago Closed 3 years ago

Perma Linux python/mozbuild/mozbuild/test/test_vendor.py::test_up_to_date_vendor TEST-UNEXPECTED-FAIL

Categories

(Firefox Build System :: Task Configuration, defect, P5)

Product:
Firefox Build System
Component:
Task Configuration
Type:
defect

Tracking

(firefox-esr91 unaffected, firefox97 unaffected, firefox98 unaffected, firefox99 fixed)

Status:
RESOLVED FIXED
Milestone:
99 Branch
Tracking Flags:
Tracking Status
firefox-esr91 --- unaffected
firefox97 --- unaffected
firefox98 --- unaffected
firefox99 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: ahal)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Attachments

(2 files)

Bug 1755569 - Fix exception in default parameters if repo url doesn't have slashes, r?#releng-reviewers!
48 bytes, text/x-phabricator-request
Details | Review
Bug 1755569 - Update vendored taskgraph to 1.1.7, r?#releng-reviewers!
48 bytes, text/x-phabricator-request
Details | Review

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=367997877&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CnsVXg3HTwCH98pxkCB7Xg/runs/0/artifacts/public/logs/live_backing.log

[task 2022-02-15T21:32:48.049Z] python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py::OSXCrossToolchainTest::test_cannot_osx_cross
[task 2022-02-15T21:32:48.049Z]   /builds/worker/checkouts/gecko/python/mozbuild/mozbuild/test/configure/test_toolchain_configure.py:375: DeprecationWarning: Please use assertEqual instead.
[task 2022-02-15T21:32:48.049Z]     self.assertEquals((var, result), (var, self.out.getvalue().strip()))
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z] -- Docs: https://docs.pytest.org/en/latest/warnings.html
[task 2022-02-15T21:32:48.049Z] ================== 54 passed, 1547 warnings in 204.80 seconds ==================
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z] Tests Completed: 100%|██████████| 77/77 [03:39<00:00, 44.85s/Test]
[task 2022-02-15T21:32:48.049Z]                                                                   
[task 2022-02-15T21:32:48.049Z] /builds/worker/checkouts/gecko/python/mozbuild/mozbuild/test/test_vendor.py
[task 2022-02-15T21:32:48.049Z] ============================= test session starts ==============================
[task 2022-02-15T21:32:48.049Z] platform linux -- Python 3.6.9, pytest-4.6.6, py-1.5.4, pluggy-0.13.1 -- /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/python-test/bin/python
[task 2022-02-15T21:32:48.049Z] rootdir: /builds/worker/checkouts/gecko, inifile: /builds/worker/checkouts/gecko/config/mozunit/mozunit/pytest.ini
[task 2022-02-15T21:32:48.049Z] collecting ... collected 1 item
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z] python/mozbuild/mozbuild/test/test_vendor.py::test_up_to_date_vendor TEST-UNEXPECTED-FAIL
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z] =================================== FAILURES ===================================
[task 2022-02-15T21:32:48.049Z] ____________________________ test_up_to_date_vendor ____________________________
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]     def test_up_to_date_vendor():
[task 2022-02-15T21:32:48.049Z]         with tempfile.TemporaryDirectory() as work_dir:
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]             def copy_to_work_dir(relative_path):
[task 2022-02-15T21:32:48.049Z]                 args = (
[task 2022-02-15T21:32:48.049Z]                     os.path.join(topsrcdir, relative_path),
[task 2022-02-15T21:32:48.049Z]                     os.path.join(work_dir, relative_path),
[task 2022-02-15T21:32:48.049Z]                 )
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]                 shutil.copytree(*args) if os.path.isdir(relative_path) else shutil.copy(
[task 2022-02-15T21:32:48.049Z]                     *args
[task 2022-02-15T21:32:48.049Z]                 )
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]             subprocess.check_call(["hg", "init", work_dir])
[task 2022-02-15T21:32:48.049Z]             os.makedirs(os.path.join(work_dir, "build"))
[task 2022-02-15T21:32:48.049Z]             os.makedirs(os.path.join(work_dir, "third_party"))
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]             # Create empty virtualenv_packages file
[task 2022-02-15T21:32:48.049Z]             with open(
[task 2022-02-15T21:32:48.049Z]                 os.path.join(work_dir, "build", "common_virtualenv_packages.txt"), "a"
[task 2022-02-15T21:32:48.049Z]             ) as file:
[task 2022-02-15T21:32:48.049Z]                 # Since VendorPython thinks "work_dir" is the topsrcdir,
[task 2022-02-15T21:32:48.049Z]                 # it will use its associated virtualenv and package configuration.
[task 2022-02-15T21:32:48.049Z]                 # Add `pip-tools` and its dependencies.
[task 2022-02-15T21:32:48.049Z]                 file.write("vendored:third_party/python/click\n")
[task 2022-02-15T21:32:48.049Z]                 file.write("vendored:third_party/python/pip\n")
[task 2022-02-15T21:32:48.049Z]                 file.write("vendored:third_party/python/pip_tools\n")
[task 2022-02-15T21:32:48.049Z]                 file.write("vendored:third_party/python/setuptools\n")
[task 2022-02-15T21:32:48.049Z]                 file.write("vendored:third_party/python/wheel\n")
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]             copy_to_work_dir(os.path.join("third_party", "python"))
[task 2022-02-15T21:32:48.049Z]             copy_to_work_dir(os.path.join("python", "mach"))
[task 2022-02-15T21:32:48.049Z]             copy_to_work_dir(os.path.join("build", "mach_virtualenv_packages.txt"))
[task 2022-02-15T21:32:48.049Z]             copy_to_work_dir(os.path.join("build", "common_virtualenv_packages.txt"))
[task 2022-02-15T21:32:48.049Z] 
[task 2022-02-15T21:32:48.049Z]             # Run the vendoring process
[task 2022-02-15T21:32:48.049Z]             vendor = VendorPython(
[task 2022-02-15T21:32:48.049Z]                 work_dir, None, Mock(), topobjdir=os.path.join(work_dir, "obj")
[task 2022-02-15T21:32:48.049Z]             )
[task 2022-02-15T21:32:48.049Z]             vendor.vendor()
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z]             # Verify that re-vendoring did not cause file changes.
[task 2022-02-15T21:32:48.050Z]             # Note that we don't want hg-ignored generated files
[task 2022-02-15T21:32:48.050Z]             # to bust the diff, so we exclude them (pycache, egg-info).
[task 2022-02-15T21:32:48.050Z]             subprocess.check_call(
[task 2022-02-15T21:32:48.050Z]                 [
[task 2022-02-15T21:32:48.050Z]                     "diff",
[task 2022-02-15T21:32:48.050Z]                     "-r",
[task 2022-02-15T21:32:48.050Z]                     os.path.join(topsrcdir, os.path.join("third_party", "python")),
[task 2022-02-15T21:32:48.050Z]                     os.path.join(work_dir, os.path.join("third_party", "python")),
[task 2022-02-15T21:32:48.050Z] >                   "--exclude=__pycache__",
[task 2022-02-15T21:32:48.050Z]                 ]
[task 2022-02-15T21:32:48.050Z]             )
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z] python/mozbuild/mozbuild/test/test_vendor.py:68:
[task 2022-02-15T21:32:48.050Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z] popenargs = (['diff', '-r', '/builds/worker/checkouts/gecko/third_party/python', '/tmp/tmp6p2emoq5/third_party/python', '--exclude=__pycache__'],)
[task 2022-02-15T21:32:48.050Z] kwargs = {}, retcode = 1
[task 2022-02-15T21:32:48.050Z] cmd = ['diff', '-r', '/builds/worker/checkouts/gecko/third_party/python', '/tmp/tmp6p2emoq5/third_party/python', '--exclude=__pycache__']
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z]     def check_call(*popenargs, **kwargs):
[task 2022-02-15T21:32:48.050Z]         """Run command with arguments.  Wait for command to complete.  If
[task 2022-02-15T21:32:48.050Z]         the exit code was zero then return, otherwise raise
[task 2022-02-15T21:32:48.050Z]         CalledProcessError.  The CalledProcessError object will have the
[task 2022-02-15T21:32:48.050Z]         return code in the returncode attribute.
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z]         The arguments are the same as for the call function.  Example:
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z]         check_call(["ls", "-l"])
[task 2022-02-15T21:32:48.050Z]         """
[task 2022-02-15T21:32:48.050Z]         retcode = call(*popenargs, **kwargs)
[task 2022-02-15T21:32:48.050Z]         if retcode:
[task 2022-02-15T21:32:48.050Z]             cmd = kwargs.get("args")
[task 2022-02-15T21:32:48.050Z]             if cmd is None:
[task 2022-02-15T21:32:48.050Z]                 cmd = popenargs[0]
[task 2022-02-15T21:32:48.050Z] >           raise CalledProcessError(retcode, cmd)
[task 2022-02-15T21:32:48.050Z] E           subprocess.CalledProcessError: Command '['diff', '-r', '/builds/worker/checkouts/gecko/third_party/python', '/tmp/tmp6p2emoq5/third_party/python', '--exclude=__pycache__']' returned non-zero exit status 1.
[task 2022-02-15T21:32:48.050Z] 
[task 2022-02-15T21:32:48.050Z] /usr/lib/python3.6/subprocess.py:311: CalledProcessError
[task 2022-02-15T21:32:48.050Z] ----------------------------- Captured stdout call -----------------------------
[task 2022-02-15T21:32:48.050Z] created virtual environment CPython3.6.9.final.0-64 in 38ms
[task 2022-02-15T21:32:48.050Z]   creator CPython3Posix(dest=/tmp/tmp6p2emoq5/obj/_virtualenvs/common, clear=False, no_vcs_ignore=False, global=False)
[task 2022-02-15T21:32:48.050Z]   activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
[task 2022-02-15T21:32:48.050Z] Collecting aiohttp==3.7.4.post0
[task 2022-02-15T21:32:48.050Z]   Downloading aiohttp-3.7.4.post0.tar.gz (1.1 MB)
[task 2022-02-15T21:32:48.050Z]   Installing build dependencies: started
[task 2022-02-15T21:32:48.050Z]   Installing build dependencies: finished with status 'done'
[task 2022-02-15T21:32:48.050Z]   Getting requirements to build wheel: started
[task 2022-02-15T21:32:48.050Z]   Getting requirements to build wheel: finished with status 'done'
[task 2022-02-15T21:32:48.050Z]     Preparing wheel metadata: started
[task 2022-02-15T21:32:48.050Z]     Preparing wheel metadata: finished with status 'done'
[task 2022-02-15T21:32:48.050Z] Collecting appdirs==1.4.4
[task 2022-02-15T21:32:48.050Z]   Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
[task 2022-02-15T21:32:48.051Z] Collecting async-timeout==3.0.1
[task 2022-02-15T21:32:48.051Z]   Using cached async_timeout-3.0.1-py3-none-any.whl (8.2 kB)
[task 2022-02-15T21:32:48.051Z] Collecting attrs==19.2.0
[task 2022-02-15T21:32:48.051Z]   Using cached attrs-19.2.0-py2.py3-none-any.whl (40 kB)
[task 2022-02-15T21:32:48.051Z] Collecting blessings==1.7
[task 2022-02-15T21:32:48.051Z]   Using cached blessings-1.7-py3-none-any.whl (18 kB)
[task 2022-02-15T21:32:48.051Z] Collecting cbor2==4.0.1
[task 2022-02-15T21:32:48.051Z]   Using cached cbor2-4.0.1-py2.py3-none-any.whl (11 kB)
[task 2022-02-15T21:32:48.051Z] Collecting certifi==2018.4.16
[task 2022-02-15T21:32:48.051Z]   Using cached certifi-2018.4.16-py2.py3-none-any.whl (150 kB)
[task 2022-02-15T21:32:48.051Z] Collecting chardet==4.0.0
[task 2022-02-15T21:32:48.051Z]   Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
[task 2022-02-15T21:32:48.051Z] Collecting click==7.1.2
[task 2022-02-15T21:32:48.051Z]   Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
[task 2022-02-15T21:32:48.051Z] Collecting colorama==0.4.4
[task 2022-02-15T21:32:48.051Z]   Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB)
[task 2022-02-15T21:32:48.051Z] Collecting compare-locales==8.2.1
[task 2022-02-15T21:32:48.051Z]   Using cached compare_locales-8.2.1-py2.py3-none-any.whl (128 kB)
[task 2022-02-15T21:32:48.051Z] Collecting cookies==2.2.1
[task 2022-02-15T21:32:48.051Z]   Using cached cookies-2.2.1-py2.py3-none-any.whl (44 kB)
[task 2022-02-15T21:32:48.051Z] Collecting coverage==5.1
[task 2022-02-15T21:32:48.051Z]   Downloading coverage-5.1.tar.gz (687 kB)
[task 2022-02-15T21:32:48.051Z] Collecting cram==0.7
[task 2022-02-15T21:32:48.051Z]   Using cached cram-0.7-py2.py3-none-any.whl (22 kB)
[task 2022-02-15T21:32:48.051Z] Collecting diskcache==4.1.0
[task 2022-02-15T21:32:48.051Z]   Using cached diskcache-4.1.0-py2.py3-none-any.whl (44 kB)
[task 2022-02-15T21:32:48.051Z] Collecting distro==1.4.0
[task 2022-02-15T21:32:48.052Z]   Using cached distro-1.4.0-py2.py3-none-any.whl (17 kB)
Flags: needinfo?(jgilbert)

Set release status flags based on info from the regressing bug 1755093

status-firefox97: --- → unaffected
status-firefox98: --- → unaffected
status-firefox99: --- → affected
status-firefox-esr91: --- → unaffected

This fails only on Linux, treeherder link.

Summary: Perma python/mozbuild/mozbuild/test/test_vendor.py::test_up_to_date_vendor TEST-UNEXPECTED-FAIL → Perma Linux python/mozbuild/mozbuild/test/test_vendor.py::test_up_to_date_vendor TEST-UNEXPECTED-FAIL
Has Regression Range: --- → yes

Only a failure on linux because This test gets skipped with pattern: os == "win" os == "mac".

Unrelated to that, I don't think this was bug 1755093. This should go to the triage owner, I guess.

Flags: needinfo?(jgilbert) → needinfo?(csabou)
Flags: needinfo?(ahal)

Oh, no this is regressed by bug 1755093 since I gave you a sloppy review and didn't notice the vendored taskgraph was being modified..

The purpose of this test is to prevent folks from making changes to files under third_party/python. Instead we'll need to fix this upstream and then re-vendor a newer version. I can handle it.

Assignee: nobody → ahal
Flags: needinfo?(ahal)

Thanks!

Here's where it first failed. Thanks Andrew for the patch.

Flags: needinfo?(csabou)
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/ci/taskgraph/rev/4050e27bd0eb Fix exception in default parameters if repo url doesn't have slashes, r=releng-reviewers,gbrown
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/79409a16bbcd Update vendored taskgraph to 1.1.7, r=releng-reviewers,gbrown

https://hg.mozilla.org/mozilla-central/rev/79409a16bbcd

Status: NEW → RESOLVED
Closed: 3 years ago
status-firefox99: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 99 Branch
See Also: → 1765416
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: