1478050 - Intermittent Mochitest harness failure OSError: [Errno 17] File exists in linux64 testing/mochitest/tests/python/test_build_profile.py::test_common_prefs_are_all_set

Reporter

Description

•

6 years ago

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=189823719&repo=autoland&lineNumber=74134

Push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=61f33f8c87508dae19c577c74ec71eae459c6aaf&selectedJob=189820943

Task details: https://tools.taskcluster.net/groups/dE3iZ2vVRTiqI_xdjGp8aA/tasks/QZeZaim7TgaZcX_5R0vO7w/details

[task 2018-07-24T15:21:52.044Z]  0:24.52 ==================================== ERRORS ====================================
[task 2018-07-24T15:21:52.044Z]  0:24.52 _______________ ERROR at setup of test_common_prefs_are_all_set ________________
[task 2018-07-24T15:21:52.045Z]  0:24.52 
[task 2018-07-24T15:21:52.045Z]  0:24.52 request = <SubRequest 'skip_using_mozinfo' for <Function 'test_common_prefs_are_all_set'>>
[task 2018-07-24T15:21:52.045Z]  0:24.53 setup_test_harness = <function inner at 0x7f3ffc002230>
[task 2018-07-24T15:21:52.046Z]  0:24.53 
[task 2018-07-24T15:21:52.046Z]  0:24.53     @pytest.fixture(autouse=True)  # noqa: F811
[task 2018-07-24T15:21:52.047Z]  0:24.53     def skip_using_mozinfo(request, setup_test_harness):
[task 2018-07-24T15:21:52.048Z]  0:24.53         """Gives tests the ability to skip based on values from mozinfo.
[task 2018-07-24T15:21:52.048Z]  0:24.53 
[task 2018-07-24T15:21:52.048Z]  0:24.53         Example:
[task 2018-07-24T15:21:52.048Z]  0:24.53             @pytest.mark.skip_mozinfo("!e10s || os == 'linux'")
[task 2018-07-24T15:21:52.048Z]  0:24.53             def test_foo():
[task 2018-07-24T15:21:52.048Z]  0:24.53                 pass
[task 2018-07-24T15:21:52.049Z]  0:24.53         """
[task 2018-07-24T15:21:52.049Z]  0:24.53 
[task 2018-07-24T15:21:52.049Z]  0:24.53 >       setup_test_harness(*setup_args)
[task 2018-07-24T15:21:52.049Z]  0:24.53 
[task 2018-07-24T15:21:52.049Z]  0:24.53 testing/mochitest/tests/python/conftest.py:102:
[task 2018-07-24T15:21:52.049Z]  0:24.53 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2018-07-24T15:21:52.049Z]  0:24.53 
[task 2018-07-24T15:21:52.049Z]  0:24.53 files_dir = '/builds/worker/checkouts/gecko/testing/mochitest/tests/python/files'
[task 2018-07-24T15:21:52.049Z]  0:24.53 args = ('mochitest', 'testing/mochitest'), kwargs = {}
[task 2018-07-24T15:21:52.049Z]  0:24.53 harness_root = '/builds/worker/fetches/tests/mochitest'
[task 2018-07-24T15:21:52.050Z]  0:24.53 test_root = '/builds/worker/fetches/tests/mochitest/tests/selftests'
[task 2018-07-24T15:21:52.050Z]  0:24.53 
[task 2018-07-24T15:21:52.050Z]  0:24.53     def inner(files_dir, *args, **kwargs):
[task 2018-07-24T15:21:52.050Z]  0:24.53         harness_root = _get_test_harness(*args, **kwargs)
[task 2018-07-24T15:21:52.050Z]  0:24.53         if harness_root:
[task 2018-07-24T15:21:52.050Z]  0:24.53             sys.path.insert(0, harness_root)
[task 2018-07-24T15:21:52.050Z]  0:24.53 
[task 2018-07-24T15:21:52.050Z]  0:24.53             # Link the test files to the test package so updates are automatically
[task 2018-07-24T15:21:52.051Z]  0:24.53             # picked up. Fallback to copy on Windows.
[task 2018-07-24T15:21:52.051Z]  0:24.53             if files_dir:
[task 2018-07-24T15:21:52.051Z]  0:24.53                 test_root = os.path.join(harness_root, 'tests', 'selftests')
[task 2018-07-24T15:21:52.051Z]  0:24.53                 if not os.path.exists(test_root):
[task 2018-07-24T15:21:52.051Z]  0:24.53                     if hasattr(os, 'symlink'):
[task 2018-07-24T15:21:52.051Z]  0:24.53 >                       os.symlink(files_dir, test_root)
[task 2018-07-24T15:21:52.051Z]  0:24.53 E                       OSError: [Errno 17] File exists
[task 2018-07-24T15:21:52.051Z]  0:24.53 
[task 2018-07-24T15:21:52.051Z]  0:24.53 testing/mozbase/moztest/moztest/selftest/fixtures.py:63: OSError
[task 2018-07-24T15:21:52.051Z]  0:24.53 =========================== 1 error in 0.10 seconds ============================

Natalia Csoregi [:nataliaCs]

Reporter

Updated

•

6 years ago

Flags: needinfo?(ahal)

Andrew Halberstadt [:ahal]

Assignee

Comment 1

•

6 years ago

Looks like this first started showing up with bug 1468812 which makes some sense I guess. I think this situation can only happen if there is a broken symbolic link, so in that case let's just remove it to be safe.

Assignee: nobody → ahal

Blocks: 1468812

Status: NEW → ASSIGNED

Flags: needinfo?(ahal)

Andrew Halberstadt [:ahal]

Assignee

Comment 2

•

6 years ago

Attached file Bug 1478050 - [moztest] Clean up broken symlinks when setting up test_harness_root — Details

It's unclear how this situation can actually happen, but it has caused a couple of
intermittents.

Phabricator Automation

Comment 3

•

6 years ago

Comment on attachment 8994825 [details]
Bug 1478050 - [moztest] Clean up broken symlinks when setting up test_harness_root

Geoff Brown [:gbrown] has approved the revision.

https://phabricator.services.mozilla.com/D2355

Attachment #8994825 - Flags: review+

Pulsebot

Comment 4

•

6 years ago

Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/dc2c0589597a
[moztest] Clean up broken symlinks when setting up test_harness_root r=gbrown

Stefan Hindli [:stefan_hindli]

Comment 5

•

6 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/dc2c0589597a

Status: ASSIGNED → RESOLVED

Closed: 6 years ago

status-firefox63: --- → fixed

Resolution: --- → FIXED

Target Milestone: --- → mozilla63

Comment hidden (Intermittent Failures Robot)

6 failures in 760 pushes (0.008 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 3
* autoland: 3

Platform breakdown:
* linux64: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-07-23&endday=2018-07-29&tree=trunk

Natalia Csoregi [:nataliaCs]

Reporter

Comment 7

•

6 years ago

Still occurring on https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=d99af6383af161ac0f0d32bc34e2027d763dc93c&selectedJob=191257598 

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=191257598&repo=autoland&lineNumber=74197

[task 2018-07-31T22:30:23.196Z]  0:25.87 
[task 2018-07-31T22:30:23.196Z]  0:25.87 testing/mochitest/tests/python/conftest.py:102:
[task 2018-07-31T22:30:23.196Z]  0:25.87 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2018-07-31T22:30:23.196Z]  0:25.87 
[task 2018-07-31T22:30:23.197Z]  0:25.87 files_dir = '/builds/worker/checkouts/gecko/testing/mochitest/tests/python/files'
[task 2018-07-31T22:30:23.197Z]  0:25.87 args = ('mochitest', 'testing/mochitest'), kwargs = {}
[task 2018-07-31T22:30:23.197Z]  0:25.87 harness_root = '/builds/worker/fetches/tests/mochitest'
[task 2018-07-31T22:30:23.197Z]  0:25.87 test_root = '/builds/worker/fetches/tests/mochitest/tests/selftests'
[task 2018-07-31T22:30:23.197Z]  0:25.87 
[task 2018-07-31T22:30:23.197Z]  0:25.87     def inner(files_dir, *args, **kwargs):
[task 2018-07-31T22:30:23.198Z]  0:25.87         harness_root = _get_test_harness(*args, **kwargs)
[task 2018-07-31T22:30:23.198Z]  0:25.87         if harness_root:
[task 2018-07-31T22:30:23.198Z]  0:25.87             sys.path.insert(0, harness_root)
[task 2018-07-31T22:30:23.198Z]  0:25.87 
[task 2018-07-31T22:30:23.198Z]  0:25.87             # Link the test files to the test package so updates are automatically
[task 2018-07-31T22:30:23.198Z]  0:25.87             # picked up. Fallback to copy on Windows.
[task 2018-07-31T22:30:23.199Z]  0:25.87             if files_dir:
[task 2018-07-31T22:30:23.199Z]  0:25.87                 test_root = os.path.join(harness_root, 'tests', 'selftests')
[task 2018-07-31T22:30:23.199Z]  0:25.87                 if not os.path.exists(test_root):
[task 2018-07-31T22:30:23.199Z]  0:25.87                     if os.path.lexists(test_root):
[task 2018-07-31T22:30:23.199Z]  0:25.87                         os.remove(test_root)
[task 2018-07-31T22:30:23.200Z]  0:25.87 
[task 2018-07-31T22:30:23.200Z]  0:25.87                     if hasattr(os, 'symlink'):
[task 2018-07-31T22:30:23.200Z]  0:25.87 >                       os.symlink(files_dir, test_root)
[task 2018-07-31T22:30:23.200Z]  0:25.87 E                       OSError: [Errno 17] File exists
[task 2018-07-31T22:30:23.200Z]  0:25.87 
[task 2018-07-31T22:30:23.201Z]  0:25.87 testing/mozbase/moztest/moztest/selftest/fixtures.py:66: OSError
[task 2018-07-31T22:30:23.201Z]  0:25.87 =========================== 1 error in 0.10 seconds ============================

Natalia Csoregi [:nataliaCs]

Reporter

Updated

•

6 years ago

Flags: needinfo?(ahal)

Andrew Halberstadt [:ahal]

Assignee

Comment 8

•

6 years ago

I have no idea how this could still be happening :/. Unless the "test_root" dir is somehow being created between the os.path.exists() check and os.symlink()..

Status: RESOLVED → REOPENED

Flags: needinfo?(ahal)

Resolution: FIXED → ---

Comment hidden (Intermittent Failures Robot)

3 failures in 2579 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 3

Platform breakdown:
* linux64: 1
* linux64-asan: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-08-06&endday=2018-08-12&tree=all

Comment hidden (Intermittent Failures Robot)

7 failures in 2672 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* autoland: 4
* mozilla-inbound: 2

Platform breakdown:
* linux64: 3
* linux64-asan: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-08-13&endday=2018-08-19&tree=all

Comment hidden (Intermittent Failures Robot)

2 failures in 2568 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* mozilla-inbound: 1

Platform breakdown:
* linux64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-08-20&endday=2018-08-26&tree=all

Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)

Updated

•

6 years ago

Summary: Mochitest harness failure OSError: [Errno 17] File exists → Intermittent Mochitest harness failure OSError: [Errno 17] File exists in linux64 testing/mochitest/tests/python/test_build_profile.py::test_common_prefs_are_all_set

Comment hidden (Intermittent Failures Robot)

1 failures in 2684 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-09-10&endday=2018-09-16&tree=all

Comment hidden (Intermittent Failures Robot)

2 failures in 2902 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1
* mozilla-inbound: 1

Platform breakdown:
* linux64-asan: 1
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-09-17&endday=2018-09-23&tree=all

Comment hidden (Intermittent Failures Robot)

1 failures in 2773 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-09-24&endday=2018-09-30&tree=all

Comment hidden (Intermittent Failures Robot)

1 failures in 2740 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* linux64-asan: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-10-08&endday=2018-10-14&tree=all

Geoff Brown [:gbrown]

Updated

•

6 years ago

Keywords: intermittent-failure

Priority: -- → P5

Comment hidden (Intermittent Failures Robot)

1 failures in 2978 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64-asan: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-10-29&endday=2018-11-04&tree=all

Comment hidden (Intermittent Failures Robot)

1 failures in 3007 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-11-05&endday=2018-11-11&tree=all

Comment hidden (Intermittent Failures Robot)

1 failures in 2387 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64-asan: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-11-19&endday=2018-11-25&tree=all

Comment hidden (Intermittent Failures Robot)

1 failures in 2808 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1478050&startday=2018-11-26&endday=2018-12-02&tree=all

Comment hidden (Intermittent Failures Robot)

Andreea Pavel [:apavel]

Updated

•

5 years ago

Status: REOPENED → RESOLVED

Closed: 6 years ago → 5 years ago

Resolution: --- → DUPLICATE

Comment hidden (Intermittent Failures Robot)