1453588 - Intermittent Travis Selenium "AssertionError: assert 2 == 3" in test_filter_jobs_by_failure_result

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P1)

Description

[Ed Morley [:emorley]]

eg:

"""
________________ test_filter_jobs_by_failure_result[exception] _________________
base_url = 'http://localhost:50540'
selenium = <selenium.webdriver.firefox.webdriver.WebDriver (session="b0663928-62b5-4153-bbfc-97b4314abd4f")>
test_jobs = [<Job: 1 test_treeherder_jobs development f1c75261017c7c5ce3000931dce4c442fe0a1297>, <Job: 2 test_treeherder_jobs deve...d54a49d763c584926377f09835c5e1a32>, <Job: 3 test_treeherder_jobs development d9cb3668f75f8f1b0b4ef503778ad0b90595a693>]
result = 'exception'
    @pytest.mark.parametrize('result', RESULTS)
    def test_filter_jobs_by_failure_result(base_url, selenium, test_jobs, result):
        page = Treeherder(selenium, base_url).open()
>       assert len(page.all_jobs) == len(RESULTS)
E       AssertionError: assert 2 == 3
E        +  where 2 = len([<pages.treeherder.Job object at 0x7fe3192ede50>, <pages.treeherder.Job object at 0x7fe318903c10>])
E        +    where [<pages.treeherder.Job object at 0x7fe3192ede50>, <pages.treeherder.Job object at 0x7fe318903c10>] = <pages.treeherder.Treeherder object at 0x7fe3191f9b90>.all_jobs
E        +  and   3 = len(['testfailed', 'busted', 'exception'])
tests/selenium/test_filter_jobs_by_failure_result.py:18: AssertionError
------------------------------- pytest-selenium --------------------------------
URL: http://localhost:50540/#/jobs?repo=test_treeherder_jobs
WARNING: Failed to gather log types: Message: GET /session/b0663928-62b5-4153-bbfc-97b4314abd4f/log/types did not match a known command
"""

(I'd paste a link to a job, but most end up being retriggered and so no longer viewable)

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3440

Comment 2

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/c6eb87308dfadac67a41115c13ccb7a610355ab5
Bug 1453588 - Improve Selenium tests by adding explicit waits