Closed Bug 1879290 Opened 2 years ago Closed 2 years ago

Perma testing/test/test_skipfails.py::test_get_failures_5 TEST-UNEXPECTED-FAIL

Tracking

(firefox124 fixed)

Status:

RESOLVED FIXED

Milestone:

124 Branch

Tracking Flags:

Tracking

Status

firefox124

---

fixed

People

(Reporter: intermittent-bug-filer, Assigned: tmarble)

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Bug 1879290 - fixes test_skipfails.py::test_get_failures_5 r=jmaher 2 years ago Tom Marble 48 bytes, text/x-phabricator-request		Details \| Review

Treeherder Bug Filer

Reporter

Description

•

2 years ago

treeherder

Filed by: pstanciu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=446470329&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/dmex8vPgQXacGNylj6ddOA/runs/0/artifacts/public/logs/live_backing.log

[task 2024-02-08T08:53:27.681Z] Tests Completed: 100%|██████████| 1/1 [00:02<00:00,  2.07s/Test]
[task 2024-02-08T08:53:27.681Z]                                                                 
[task 2024-02-08T08:53:27.681Z] /builds/worker/checkouts/gecko/testing/test/test_skipfails.py
[task 2024-02-08T08:53:27.682Z] 2024-02-08 08:53:26.191 | WARNING  | mozci.configuration:__init__:149 - Configuration path /builds/worker/.config/mozci/config.toml is not a file.
[task 2024-02-08T08:53:27.682Z] 08:53:26.242 | INFO     | mozci.data.base:50  - Sources selected, in order of priority: ('hgmo', 'taskcluster', 'treeherder_client', 'errorsummary', 'bugbug').
[task 2024-02-08T08:53:27.682Z] ============================= test session starts ==============================
[task 2024-02-08T08:53:27.682Z] platform linux -- Python 3.8.10, pytest-7.0.1, pluggy-1.4.0 -- /builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/python-test/bin/python
[task 2024-02-08T08:53:27.682Z] rootdir: /builds/worker/checkouts/gecko, configfile: config/mozunit/mozunit/pytest.ini
[task 2024-02-08T08:53:27.683Z] plugins: mock-3.12.0
[task 2024-02-08T08:53:27.683Z] collecting ... collected 12 items
[task 2024-02-08T08:53:27.683Z] 
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_revision PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_tasks PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_failures_1 PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_failures_2 PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_failures_3 PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_failures_4 PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_failures_5 TEST-UNEXPECTED-FAIL
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_bug_by_id PASSED
[task 2024-02-08T08:53:27.683Z] testing/test/test_skipfails.py::test_get_variants PASSED
[task 2024-02-08T08:53:27.684Z] testing/test/test_skipfails.py::test_task_to_skip_if PASSED
[task 2024-02-08T08:53:27.684Z] testing/test/test_skipfails.py::test_get_filename_in_manifest PASSED
[task 2024-02-08T08:53:27.684Z] testing/test/test_skipfails.py::test_label_to_platform_testname PASSED
[task 2024-02-08T08:53:27.684Z] 
[task 2024-02-08T08:53:27.684Z] =================================== FAILURES ===================================
[task 2024-02-08T08:53:27.684Z] _____________________________ test_get_failures_5 ______________________________
[task 2024-02-08T08:53:27.684Z] testing/test/test_skipfails.py:89: in test_get_failures_5
[task 2024-02-08T08:53:27.685Z]     get_failures("wayland-tasks-5.json", "wayland-failures-5.json")
[task 2024-02-08T08:53:27.685Z] testing/test/test_skipfails.py:64: in get_failures
[task 2024-02-08T08:53:27.685Z]     assert actual_failures == expected_failures
[task 2024-02-08T08:53:27.686Z] E   assert ('{\n'\n '  "browser/components/places/tests/browser/browser.toml": {\n'\n '    "label": {\n'\n '      "test-linux2204-64-wayland/opt-mochitest-browser-chrome-swr-3": {\n'\n '        "durations": {\n'\n '          "Bgc6We1sSjakIo3V9crldw": 1521792,\n'\n '          "EH0WPmGtQsGzu5K7OxFOyg": 1407021,\n'\n '          "bxDhZzsxQw65gn-veX1PHg": 1320318\n'\n '        },\n'\n '        "median_duration": 1407021,\n'\n '        "opt": false,\n'\n '        "path": {\n'\n '          '\n '"browser/components/places/tests/browser/browser_bookmarkProperties_addKeywordForThisSearch.js": '\n '{\n'\n '            "classification": "disable_recommended",\n'\n '            "failed_runs": 3,\n'\n '            "runs": {\n'\n '              "Bgc6We1sSjakIo3V9crldw": false,\n'\n '              "EH0WPmGtQsGzu5K7OxFOyg": false,\n'\n '              "bxDhZzsxQw65gn-veX1PHg": false\n'\n '            },\n'\n '            "total_runs": 3\n'\n '          },\n'\n '          '\n '"browser/components/places/tests/browser/browser_bookmark_popup.js": {\n'\n '            "classification": "secondary",\n'\n '            "failed_runs": 3,\n'\n '            "runs": {\n'\n '              "Bgc6We1sSjakIo3V9crldw": false,\n'\n '              "EH0WPmGtQsGzu5K7OxFOyg": false,\n'\n '              "bxDhZzsxQw65gn-veX1PHg": false\n'\n '            },\n'\n '            "total_runs": 3\n'\n '          }\n'\n '        },\n'\n '        "sum_by_label": {\n'\n '          "disable_manifest": 0,\n'\n '          "disable_recommended": 1,\n'\n '          "disable_too_long": 0,\n'\n '          "intermittent": 0,\n'\n '          "secondary": 1,\n'\n '          "success": 0,\n'\n '          "unknown": 0\n'\n '        },\n'\n '        "total_duration": 4249131\n'\n '      }\n'\n '    }\n'\n '  }\n'\n '}') == ('{\n'\n '  "browser/components/places/tests/browser/browser.toml": {\n'\n '    "label": {\n'\n '      "test-linux2204-64-wayland/opt-mochitest-browser-chrome-swr-3": {\n'\n '        "durations": {\n'\n '          "Bgc6We1sSjakIo3V9crldw": 1521792,\n'\n '          "EH0WPmGtQsGzu5K7OxFOyg": 1407021,\n'\n '          "bxDhZzsxQw65gn-veX1PHg": 1320318\n'\n '        },\n'\n '        "median_duration": 1407021,\n'\n '        "opt": true,\n'\n '        "path": {\n'\n '          "DEFAULT": {\n'\n '            "classification": "disable_too_long",\n'\n '            "failed_runs": 1,\n'\n '            "runs": {\n'\n '              "bxDhZzsxQw65gn-veX1PHg": false\n'\n '            },\n'\n '            "total_runs": 1\n'\n '          },\n'\n '          '\n '"browser/components/places/tests/browser/browser_bookmarkProperties_addKeywordForThisSearch.js": '\n '{\n'\n '            "classification": "disable_recommended",\n'\n '            "failed_runs": 3,\n'\n '            "runs": {\n'\n '              "Bgc6We1sSjakIo3V9crldw": false,\n'\n '              "EH0WPmGtQsGzu5K7OxFOyg": false,\n'\n '              "bxDhZzsxQw65gn-veX1PHg": false\n'\n '            },\n'\n '            "total_runs": 3\n'\n '          },\n'\n '          '\n '"browser/components/places/tests/browser/browser_bookmark_popup.js": {\n'\n '            "classification": "secondary",\n'\n '            "failed_runs": 3,\n'\n '            "runs": {\n'\n '              "Bgc6We1sSjakIo3V9crldw": false,\n'\n '              "EH0WPmGtQsGzu5K7OxFOyg": false,\n'\n '              "bxDhZzsxQw65gn-veX1PHg": false\n'\n '            },\n'\n '            "total_runs": 3\n'\n '          }\n'\n '        },\n'\n '        "sum_by_label": {\n'\n '          "disable_manifest": 0,\n'\n '          "disable_recommended": 1,\n'\n '          "disable_too_long": 1,\n'\n '          "intermittent": 0,\n'\n '          "secondary": 1,\n'\n '          "success": 0,\n'\n '          "unknown": 0\n'\n '        },\n'\n '        "total_duration": 4249131\n'\n '      }\n'\n '    }\n'\n '  }\n'\n '}')
[task 2024-02-08T08:53:27.686Z] E       {
[task 2024-02-08T08:53:27.686Z] E         "browser/components/places/tests/browser/browser.toml": {
[task 2024-02-08T08:53:27.686Z] E           "label": {
[task 2024-02-08T08:53:27.686Z] E             "test-linux2204-64-wayland/opt-mochitest-browser-chrome-swr-3": {
[task 2024-02-08T08:53:27.686Z] E               "durations": {
[task 2024-02-08T08:53:27.686Z] E                 "Bgc6We1sSjakIo3V9crldw": 1521792,
[task 2024-02-08T08:53:27.686Z] E                 "EH0WPmGtQsGzu5K7OxFOyg": 1407021,
[task 2024-02-08T08:53:27.686Z] E                 "bxDhZzsxQw65gn-veX1PHg": 1320318
[task 2024-02-08T08:53:27.686Z] E               },
[task 2024-02-08T08:53:27.686Z] E               "median_duration": 1407021,
[task 2024-02-08T08:53:27.686Z] E     -         "opt": true,
[task 2024-02-08T08:53:27.686Z] E     ?                ^^^
[task 2024-02-08T08:53:27.687Z] E     +         "opt": false,
[task 2024-02-08T08:53:27.687Z] E     ?                ^^^^
[task 2024-02-08T08:53:27.687Z] E               "path": {
[task 2024-02-08T08:53:27.687Z] E     -           "DEFAULT": {
[task 2024-02-08T08:53:27.687Z] E     -             "classification": "disable_too_long",
[task 2024-02-08T08:53:27.687Z] E     -             "failed_runs": 1,
[task 2024-02-08T08:53:27.687Z] E     -             "runs": {
[task 2024-02-08T08:53:27.687Z] E     -               "bxDhZzsxQw65gn-veX1PHg": false
[task 2024-02-08T08:53:27.687Z] E     -             },
[task 2024-02-08T08:53:27.687Z] E     -             "total_runs": 1
[task 2024-02-08T08:53:27.687Z] E     -           },
[task 2024-02-08T08:53:27.687Z] E                 "browser/components/places/tests/browser/browser_bookmarkProperties_addKeywordForThisSearch.js": {
[task 2024-02-08T08:53:27.687Z] E                   "classification": "disable_recommended",
[task 2024-02-08T08:53:27.687Z] E                   "failed_runs": 3,
[task 2024-02-08T08:53:27.688Z] E                   "runs": {
[task 2024-02-08T08:53:27.688Z] E                     "Bgc6We1sSjakIo3V9crldw": false,
[task 2024-02-08T08:53:27.688Z] E                     "EH0WPmGtQsGzu5K7OxFOyg": false,
[task 2024-02-08T08:53:27.688Z] E                     "bxDhZzsxQw65gn-veX1PHg": false
[task 2024-02-08T08:53:27.688Z] E                   },
[task 2024-02-08T08:53:27.688Z] E                   "total_runs": 3
[task 2024-02-08T08:53:27.688Z] E                 },
[task 2024-02-08T08:53:27.688Z] E                 "browser/components/places/tests/browser/browser_bookmark_popup.js": {
[task 2024-02-08T08:53:27.688Z] E                   "classification": "secondary",
[task 2024-02-08T08:53:27.688Z] E                   "failed_runs": 3,
[task 2024-02-08T08:53:27.688Z] E                   "runs": {
[task 2024-02-08T08:53:27.688Z] E                     "Bgc6We1sSjakIo3V9crldw": false,
[task 2024-02-08T08:53:27.688Z] E                     "EH0WPmGtQsGzu5K7OxFOyg": false,
[task 2024-02-08T08:53:27.688Z] E                     "bxDhZzsxQw65gn-veX1PHg": false
[task 2024-02-08T08:53:27.689Z] E                   },
[task 2024-02-08T08:53:27.689Z] E                   "total_runs": 3
[task 2024-02-08T08:53:27.689Z] E                 }
[task 2024-02-08T08:53:27.689Z] E               },
[task 2024-02-08T08:53:27.689Z] E               "sum_by_label": {
[task 2024-02-08T08:53:27.689Z] E                 "disable_manifest": 0,
[task 2024-02-08T08:53:27.689Z] E                 "disable_recommended": 1,
[task 2024-02-08T08:53:27.689Z] E     -           "disable_too_long": 1,
[task 2024-02-08T08:53:27.689Z] E     ?                               ^
[task 2024-02-08T08:53:27.689Z] E     +           "disable_too_long": 0,
[task 2024-02-08T08:53:27.689Z] E     ?                               ^
[task 2024-02-08T08:53:27.689Z] E                 "intermittent": 0,
[task 2024-02-08T08:53:27.689Z] E                 "secondary": 1,
[task 2024-02-08T08:53:27.689Z] E                 "success": 0,
[task 2024-02-08T08:53:27.690Z] E                 "unknown": 0
[task 2024-02-08T08:53:27.690Z] E               },
[task 2024-02-08T08:53:27.690Z] E               "total_duration": 4249131
[task 2024-02-08T08:53:27.690Z] E             }
[task 2024-02-08T08:53:27.690Z] E           }
[task 2024-02-08T08:53:27.690Z] E         }
[task 2024-02-08T08:53:27.690Z] E       }
[task 2024-02-08T08:53:27.690Z] ----------------------------- Captured stderr call -----------------------------
[task 2024-02-08T08:53:27.690Z] WARNING: Task Bgc6We1sSjakIo3V9crldw no longer exists.
[task 2024-02-08T08:53:27.690Z] ========================= 1 failed, 11 passed in 1.25s =========================
[task 2024-02-08T08:53:27.690Z] 
[task 2024-02-08T08:53:27.690Z] 
[task 2024-02-08T08:53:27.690Z] Tests Completed: 100%|██████████| 1/1 [00:02<00:00,  2.08s/Test]Return code from mach python-test: 1
[task 2024-02-08T08:53:27.720Z] 
[taskcluster 2024-02-08 08:53:28.037Z] === Task Finished ===
[taskcluster 2024-02-08 08:53:28.037Z] Unsuccessful task run with exit code: 1 completed in 40.551 seconds

pstanciu

Comment 1

•

2 years ago

Hi Joel, it also fails on previously green tasks, could you please take a look at this?
Thank you!

Flags: needinfo?(jmaher)

Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)

Comment 2

•

2 years ago

Also do you know why the skip-fail task got scheduled? https://hg.mozilla.org/integration/autoland/rev/dd1c85f56d07c7aa09d9407ae76c50d6928802f5 changed as taskcluster task definition fail for document generation, and skip-fail should only run when specific parts of the testing folder change.

Tom Marble

Assignee

Comment 3

•

2 years ago

Attached file Bug 1879290 - fixes test_skipfails.py::test_get_failures_5 r=jmaher — Details

Phabricator Automation

Updated

•

2 years ago

Assignee: nobody → tmarble

Status: NEW → ASSIGNED

Joel Maher ( :jmaher ) (UTC -8)

Updated

•

2 years ago

Flags: needinfo?(jmaher)

Pulsebot

Comment 4

•

2 years ago

Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/20970ba677c0 fixes test_skipfails.py::test_get_failures_5 r=jmaher

Atila Butkovits

Comment 5

•

2 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/20970ba677c0

Status: ASSIGNED → RESOLVED

Closed: 2 years ago

status-firefox124: --- → fixed

Resolution: --- → FIXED

Target Milestone: --- → 124 Branch

Comment hidden (Intermittent Failures Robot)

You need to log in before you can comment on or make changes to this bug.

Bugzilla

Perma testing/test/test_skipfails.py::test_get_failures_5 TEST-UNEXPECTED-FAIL

Categories

(Firefox Build System :: Task Configuration, defect, P5)

Tracking

(firefox124 fixed)

People

(Reporter: intermittent-bug-filer, Assigned: tmarble)

References

Details

(Keywords: intermittent-failure)

Crash Data

Security

(public)

User Story

Attachments

(1 file)

Description

Comment 1

Comment 2

Comment 3

Updated

Updated

Comment 4

Comment 5

Comment 6

Attachment

General

Description

File Name

Content Type