Open Bug 1488837 Opened 4 years ago Updated 2 months ago
Test fails with headless, passes without
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:61.0) Gecko/20100101 Firefox/61.0 Build ID: 20180807170231 Steps to reproduce: Running the test (incl. file with bug.py) on Ubuntu using Firefox in headless mode, will result in a failure. All things equal, without headless, test passes. I've tested running it against: - TestingBot (Ubuntu/FF59) - selenium/standalone-firefox-debug:3.14.0-arsenic/curium docker (FF61.0.1) using: - my local machine (macosx) - Bitbucket Pipelines All with the same result, fail w/ headless - pass w/o. All instances have used Geckodriver 0.21.0. Actual results: Test fails on Linux using headless. Expected results: Test should pass. :)
Should mention that in all instances selenium python bindings was 3.14.0 In the TestingBot case Selenium Grid was 3.13.0
I can replicate this on an Ubuntu 14.04 machine. The traceback is: python ~/Downloads/bug.py Traceback (most recent call last): File "/home/dave/Downloads/bug.py", line 81, in <module> exit(run_it()) File "/home/dave/Downloads/bug.py", line 71, in run_it get_element(value).click() File "/home/dave/Downloads/bug.py", line 12, in get_element return WebDriverWait(driver, timeout).until(exp_cond((By.CSS_SELECTOR, locator))) File "/home/dave/.bug1488837/local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Looking at the script, this suggests that the click on the first dropdown hasn't worked? As reported, it passes when run without headless set, and it passes with/without headless for me on macOS.
Dave, did you create a minimized HTML testcase? Would be good to just have that listed here.
(In reply to Henrik Skupin (:whimboo) from comment #3) > Dave, did you create a minimized HTML testcase? Would be good to just have > that listed here. No, I just used Jim's. It's only requires the selenium dependency.
Attachment #9006633 - Attachment mime type: text/x-python-script → text/plain
Dave, the problem you are seeing here is because in some cases the initial page load is not finishing. When running the script outside of headless you can see that the spinner is spinning forever. I can always replicate that with a developer build of Firefox but not with a Nightly build. When I change the attached script to customize the Firefox binary the test works fine in both modes: > from selenium.webdriver.firefox.options import Options > > opts = Options() > opts.binary = "/Applications/FirefoxNightly.app/Contents/MacOS/firefox" > > driver = webdriver.Firefox(options=opts) Jim, please attach a trace level log to this bug so that I can see which step is failing for you. See https://firefox-source-docs.mozilla.org/testing/geckodriver/geckodriver/TraceLogs.html in how to get one.
Here's the trace for bug.py: https://eulogtestingbot.s3-eu-west-1.amazonaws.com/6fc237f4-b453-49b3-a3b0-1dbb6c2e5710_firefox.txt?X-Amz-Expires=3600&X-Amz-Date=20180910T203602Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIQ3ZQKDOJEMHVEAA/20180910/eu-west-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=b5c471b7b9627b39f9fd133226bcbeeed1adf2fd03903a08252a26d432ecf493 Here's the trace for the ACTUAL test: https://eulogtestingbot.s3-eu-west-1.amazonaws.com/c29a8eff-05af-411e-8606-c6c66a2878e7_firefox.txt?X-Amz-Expires=3600&X-Amz-Date=20180910T202830Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIQ3ZQKDOJEMHVEAA/20180910/eu-west-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=63e9c39086875ec62a62223602198228d88d0698af68c5a811bf8cb706ebbc6d Hope that helps!
Here's a link to the TestingBot job for bug.py: https://testingbot.com/tests/6fc237f4-b453-49b3-a3b0-1dbb6c2e5710 Here's a link to the TestingBot job for the real test: https://testingbot.com/tests/c29a8eff-05af-411e-8606-c6c66a2878e7 As far as inifinite spinner goes, that's not what I'm seeing, in either test. Not according to the screenshots at least. For some reason, the screenshots isn't working on TB for headless. I will raise an issue there. They do work and are added to the html-report.
Ok, the hang for the spinner I can workaround by using a Nightly build of Firefox. But when I then run your script without headless, it hangs in the following line: > get_element(first_dropdown).click() Jim, do you have a chance to create a minimized HTML testcase for your issue? As long as I cannot even run the test outside of headless there is nothing I can do. Do you have a testingbot video for a non-headless job? I would like to see what it's actually doing.
(In reply to jim.brannlund from comment #6) > Here's the trace for bug.py: [..] > Here's the trace for the ACTUAL test: [..] Both links are not accessible.
Priority: -- → P3
Summary: Test fails with headless, passes without. → Test fails with headless, passes without
You need to log in before you can comment on or make changes to this bug.