1886230 - Perma taskcluster/test/test_mach_try_auto.py::test_tasks_are_scheduled[mochitest-browser-chrome] TEST-UNEXPECTED-FAIL

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=451319752&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EU_3rUhgQZOkTkLCsVT7gA/runs/0/artifacts/public/logs/live_backing.log

---

taskcluster/test/test_generate_params.py::test_generate_graphs[mb-push-firefox] PASSED
[task 2024-03-19T16:08:58.265Z] 
[task 2024-03-19T16:08:58.265Z] ======================= 30 passed in 1401.08s (0:23:21) ========================
[task 2024-03-19T16:08:58.265Z] 
[task 2024-03-19T16:08:58.265Z] 
[task 2024-03-19T16:08:58.266Z] Tests Completed: 100%|██████████| 6/6 [23:21<00:00, 276.62s/Test]
[task 2024-03-19T16:08:58.266Z]                                                                  
[task 2024-03-19T16:08:58.266Z] /builds/worker/checkouts/gecko/taskcluster/test/test_mach_try_auto.py
[task 2024-03-19T16:08:58.266Z] ============================= test session starts ==============================
[task 2024-03-19T16:08:58.266Z] platform linux -- Python 3.8.10, pytest-7.0.1, pluggy-1.4.0 -- /builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/python-test/bin/python
[task 2024-03-19T16:08:58.266Z] rootdir: /builds/worker/checkouts/gecko, configfile: config/mozunit/mozunit/pytest.ini
[task 2024-03-19T16:08:58.266Z] plugins: mock-3.12.0
[task 2024-03-19T16:08:58.266Z] collecting ... collected 8 items
[task 2024-03-19T16:08:58.266Z] 
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_generate_graph PASSED
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_only_important_manifests PASSED
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_tasks_are_scheduled[mochitest-browser-chrome] TEST-UNEXPECTED-FAIL
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_tasks_are_not_scheduled[no shippable builds] PASSED
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_tasks_are_not_scheduled[no fuzzing builds] PASSED
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_tasks_are_not_scheduled[no ccov builds] PASSED
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_tasks_are_not_scheduled[no build-signing] TEST-UNEXPECTED-FAIL
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py::test_tasks_are_not_scheduled[no upload-symbols] PASSED
[task 2024-03-19T16:08:58.266Z] 
[task 2024-03-19T16:08:58.266Z] =================================== FAILURES ===================================
[task 2024-03-19T16:08:58.266Z] ______________ test_tasks_are_scheduled[mochitest-browser-chrome] ______________
[task 2024-03-19T16:08:58.266Z] taskcluster/test/test_mach_try_auto.py:71: in test_tasks_are_scheduled
[task 2024-03-19T16:08:58.266Z]     assert len(tasks) >= min_expected
[task 2024-03-19T16:08:58.266Z] E   assert 0 >= 5
[task 2024-03-19T16:08:58.266Z] E    +  where 0 = len([])
[task 2024-03-19T16:08:58.266Z] ________________ test_tasks_are_not_scheduled[no build-signing] ________________
[task 2024-03-19T16:08:58.266Z] [XPASS(strict)] some xpcshell tests require signed builds
[task 2024-03-19T16:08:58.267Z] =================== 2 failed, 6 passed in 470.76s (0:07:50) ====================
[task 2024-03-19T16:08:58.267Z] 
[task 2024-03-19T16:08:58.267Z] 
[task 2024-03-19T16:08:58.267Z] Tests Completed: 100%|██████████| 6/6 [23:21<00:00, 233.65s/Test]Return code from mach python-test: 1
[task 2024-03-19T16:08:58.293Z] 
[taskcluster 2024-03-19 16:08:58.763Z] === Task Finished ===
[taskcluster 2024-03-19 16:08:58.763Z] Unsuccessful task run with exit code: 1 completed in 1560.337 seconds

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

:ahal, since you are the author of the regressor, bug 1884364, could you take a look?

For more information, please visit BugBot documentation.

Comment 2

[Andrew Halberstadt [:ahal]]

The problem is that the get_locally_changed_files function returns an empty list when running from a public commit (which makes total sense). It also turns out that my "fix" for this in https://hg.mozilla.org/integration/autoland/rev/dc701f4ef6c4 is actually a no-op, as we were previously defaulting to using get_locally_changed_files anyway (I got mixed up and thought these tests were using the parameter files under taskcluster/test/params). This explains why the test passes on try but fails on autoland (because try is non-publishing, so it does have outgoing changes compared to central).

I think the fix here is going to be to hardcode some fake files_changed into the tests.

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1886230 - Backed out changeset dc701f4ef6c4, r?#taskgraph-reviewers!

This didn't fix the problem and was actually a no-op.

Comment 4

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1886230 - [ci] Add dummy 'files_changed' value to test_mach_try_auto.py, r?#taskgraph-reviewers!

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1884364

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ec79f7deb4bc
Backed out changeset dc701f4ef6c4, r=taskgraph-reviewers,bhearsum
https://hg.mozilla.org/integration/autoland/rev/37976874eb0c
[ci] Add dummy 'files_changed' value to test_mach_try_auto.py, r=taskgraph-reviewers,bhearsum

Comment 7

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/ec79f7deb4bc
https://hg.mozilla.org/mozilla-central/rev/37976874eb0c

Comment 8

[Intermittent Failures Robot]

16 failures in 3791 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 14
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64: 16
  • opt: 16

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1886230&startday=2024-03-18&endday=2024-03-24&tree=all