Closed Bug 1538196 Opened 6 years ago Closed 6 years ago

NetworkError when attempting to fetch resource starting with Firefox 63

Categories

(Core :: Networking, defect)

63 Branch
defect
Not set
normal

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: jeremy.cattau, Unassigned, NeedInfo)

Details

(Keywords: regression)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36

Steps to reproduce:

I submitted this as a bug against geckodriver that got closed immediately and was told to log it against Firefox. https://github.com/mozilla/geckodriver/issues/1529 Admittedly, I know I didn't provide much to go on but I'm not really sure what kind of information I can provide. I'll try to be more in depth here.

We are using Geb (a wrapper on top of WebDriver) for automated UI testing for our companies web applications. We use self-signed certificates for our Dev and Test environments but our Production environment uses a paid vendor certificate. Recently I discovered that while trying to test one of our applications in our Test environment that Firefox was failing a bunch of tests after they were working perfectly on Chrome.

The tests start up and login in just fine, but after I log in and get to our homepage, the page acts like it never finishes loading and I see a "Transferring from <server>" message in the bottom left that just stays there. It never times out and the tests eventually fail after a long wait time. This is obviously a problem for us.

I tested with geckodriver 0.22.0, 0.23.0, and 0.24.0 against several versions of Firefox. Firefox 62 (.0.1 and .0.2) all work just fine. I do not experience this issue at all on those versions and before. Firefox 63 appears to be when this broke and has persisted up through Firefox 66 and even the beta build. There appears to be something introduced into Firefox 63 that is causing this problem.

The thing is, and the reason I logged this against geckodriver, is because I don't have this issue when I launch the page manually. It appears to only happen through automation/WebDriver.

Actual results:

The console output I get from Selenium/geckodriver is this (Mind you this is 5 browsers in parallel):

console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: http://%(server)s/dummy/blocklist//buckets/monitor/collections/changes/records?_expected=%221553207848478%22 is not a valid URL..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:203:13

console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: http://%(server)s/dummy/blocklist//buckets/monitor/collections/changes/records?_expected=%221553207848478%22 is not a valid URL..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:203:13

console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: http://%(server)s/dummy/blocklist//buckets/monitor/collections/changes/records?_expected=%221553207848478%22 is not a valid URL..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:203:13

console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: http://%(server)s/dummy/blocklist//buckets/monitor/collections/changes/records?_expected=%221553207848478%22 is not a valid URL..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resocurce://serviceosnsole.error:- settingsBro/aremote-settindgsca.jss:203:13
g@https://www-t.companyname.com/wps/contenthandler/s000/!ut/p/digest!I09QqdexEzXA4cnnhWD_QQ/war/GroupTheme85S/themes/groupTheme85/js/tealeaf.js:9:48614
sendRequest@https://www-t.companyname.com/wps/contenthandler/s000/!ut/p/digest!I09QqdexEzXA4cnnhWD_QQ/war/GroupTheme85S/themes/groupTheme85/js/tealeaf.js:9:4t9324
y@https://www-t.companyname.Scom/wps/ercontenvthandice:
ler/s000/!ut/p/digest!I09QqdexEzXA4cnnhWD_QQ/war/GroupTheme85S/themes/groupTheme85/js/tealeaf.js:9:31963
e@https://www-t.companyname.co m/wps/ conrtenteceivedBrhoadcasatndlMessera/s0g0e:0 h/!ut/pa/nddler for
remote-settings/monitor_changes
roupTheme85S/themes/groupThemhWD_QQ/ erwar/ror:G
Message: Error: Polling for changes failed: http://%(server)s/deummy/blocklist//buckets/monitor/col8lections/changes/records?_expected=%221553207848478%22 is not a valid URL..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-setting5s.js:203:13

Expected results:

Firefox should not hang on "Transferring data from <server>". Issue started in Firefox 63 and has persisted through 66beta.

I am not sure what I can do to help but I am happy to provide any kind of logs or anything, I just need to know how to assist.

I also forgot to mention this is happening on Windows only. Mac Firefox 66 works just fine. Windows does not. I am using 64-bit.

Component: Untriaged → Networking
Product: Firefox → Core
Has STR: --- → yes

Jc, I'll ask you for narrowing down the regression range. You can use https://mozilla.github.io/mozregression/ for that, but if your setup is more complicated, you will probably have to bisect manually. Note that mozregression allows you to download-only installation bundles.

I suspect some URL related changes we have made in that version, perhaps, but it would be nice to be sure.

When you look at the error in the console, http://%(server)s/ is really not a valid host name. I have no idea how geckodriver works, but this may have been a problem there.

Let's see what change caused this at the first place.

Thanks.

Flags: needinfo?(jeremy.cattau)

Honza,

With our setup, I will have to do it manually I believe. How would I go about that? Do I just start downloading nightly builds in the suspected range and go through one by one until I find the culprit? This may take me some time to accomplish.

Hello jc,
mozregression gives you a binary search so it won't take much time if it's reliably to reproduce.

Hi Jeremy, we are unable to progress here without a regression range.
If you are able to help us with this we'll reopen the bug. Thanks!

Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → INVALID

I confirm the same bug with firefox 66.0.4+build3-0ubuntu0.16.04.1 on ubuntu16.04 (inside https://hub.docker.com/r/ypasmk/robot-framework docker. I updated both geckodriver (v 0.24.0) and firefox (66.0.4).

Running a simple python script:

from selenium import webdriver
from os import getcwd
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# configure firefox profile to automatically save csv files in the current directory
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.download.dir", getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv")
driver = webdriver.Firefox(firefox_profile=fp, executable_path=r'/usr/bin/geckodriver')
driver.get("https://www.thinkbroadband.com/download")
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='specific-download-headline' and contains(., 'Extra Small File (5MB)')]//following::p[1]/a"))).click()
driver.save_screenshot("/screenshot.png")

like this: xvfb-run python script.py works fine with geckodriver.log:

1557405726699   mozrunner::runner       INFO    Running command: "/usr/bin/firefox" "-marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.RfXzMncAJBCi"
1557405727221   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons                                 
1557405727222   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/                            
1557405727222   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*

(firefox:5128): LIBDBUSMENU-GLIB-WARNING **: Unable to get session bus: Failed to execute child process "dbus-launch" (No such file or directory)
1557405728522   Marionette      INFO    Listening on port 44307
1557405728624   Marionette      WARN    TLS certificate errors will be ignored for this session

however running our outomated tests with pybot (robot-framework) with the following code:

Run Firefox
    [Documentation]    *Creating the driver for Firefox in Headless mode*
    ...
    ${firefox options}=    Evaluate    sys.modules['selenium.webdriver'].firefox.webdriver.Options()    sys, selenium.webdriver
    Call Method    ${firefox options}    add_argument    -headless
    Create Webdriver    Firefox    firefox_options=${firefox options}
    Set Window Size    1600    900

with command xvfb-run pybot path/to/tests gives geckodriver.log:

1557408510778   geckodriver     INFO    Listening on 127.0.0.1:42043
1557408511791   mozprofile::profile     INFO    Using profile path /tmp/rust_mozprofile.zHmJrEw2l68v
1557408511794   geckodriver::marionette INFO    Starting browser /usr/bin/firefox
1557408511799   geckodriver::marionette INFO    Connecting to Marionette on localhost:37691
*** You are running in headless mode.
1557408512308   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1557408512308   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1557408512308   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1557408513665   Marionette      INFO    Listening on port 2828
console.error: BroadcastService: 
  receivedBroadcastMessage: handler for
  remote-settings/monitor_changes
  threw error:
  Message: Error: Polling for changes failed: NetworkError when attempting to fetch resource..
  Stack:
    remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:190:13

Do you think you could use mozregression to find which commit introduced the bug?
https://mozilla.github.io/mozregression/install.html
https://mozilla.github.io/mozregression/documentation/automatic-bisection.html

Thanks!

Flags: needinfo?(chrismaes87)

During my lasts tests the error still pops up, but geckodriver is working though.

I did this on working hours, and since the tests are working now, I cannot spend the time to dig into it. Sorry.

Flags: needinfo?(chrismaes87)

I confirm this issue with below env:

OS: Debian docker container
Selenium driver version: 3.11.0
Geckodriver version: v0.24.0
Firefox version = 67.0.3

Reproduction steps:

  1. Run a docker container with base images as FROM debian:stretch ( with basic packages fo linux and above)
  2. Run a python script foo.py as :
    from selenium import webdriver
    from selenium.webdriver import DesiredCapabilities

firefox_options = webdriver.FirefoxOptions()
firefox_options.log.level = "trace"
firefox_options.binary = '/usr/bin/firefox'
firefox_options.add_argument('--headless')
firefox_options.add_argument('--disable-gpu')
firefox_options.add_argument('--no-sandbox')
firefox_options.add_argument('window-size=1200,1100')
d = webdriver.Firefox(executable_path='/usr/bin/geckodriver', options=firefox_options)
d.get("https://google.com")
print("=========END========")

Observation : same as mentioned by @chrismaes87. Firefox session get hanged on listening port
Workaround : downgraded firefox version to 62.0.2 as mentioned by @jc

Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: INVALID → ---

Hi Arvind,

Can you also check with Firefox 67? I am 90% sure this was already fixed.
Otherwise it would also be great if you were able to find the patch that caused the regression using the tool I linked in comment 8

Flags: needinfo?(arvind1843)

I have already tried with 67.0.3 and its not working.Can you give me today time and I will respond with your comment-8 debug result. Also if you want I can give you dockerfile that would be a amin job for you to reproduce with any particular version

Flags: needinfo?(arvind1843)

(In reply to Valentin Gosu [:valentin] (he/him) from comment #11)

Hi Arvind,

Can you also check with Firefox 67? I am 90% sure this was already fixed.
Otherwise it would also be great if you were able to find the patch that caused the regression using the tool I linked in comment 8
hello,

I should have commented here . Sorry for that.

I have already tried with 67.0.3 and its not working.Can you give me today time and I will respond with your comment-8 debug result. Also if you want I can give you dockerfile that would be a amin job for you to reproduce with any particular version.

(In reply to Arvind Kumar patel from comment #13)

(In reply to Valentin Gosu [:valentin] (he/him) from comment #11)
I have already tried with 67.0.3 and its not working.

Thanks for checking. That's good to know.

Can you give me today time and I will respond with your comment-8 debug result. Also if you want I can give you dockerfile that would be a amin job for you to reproduce with any particular version.

If you can find the regression range that would be very helpful.
Otherwise please post the docker file and we'll try to reproduce it sometimes in the next week. Thanks!

(In reply to Valentin Gosu [:valentin] (he/him) from comment #14)

(In reply to Arvind Kumar patel from comment #13)

(In reply to Valentin Gosu [:valentin] (he/him) from comment #11)
I have already tried with 67.0.3 and its not working.

Thanks for checking. That's good to know.

Can you give me today time and I will respond with your comment-8 debug result. Also if you want I can give you dockerfile that would be a amin job for you to reproduce with any particular version.

If you can find the regression range that would be very helpful.
Otherwise please post the docker file and we'll try to reproduce it sometimes in the next week. Thanks!

Sure. I have added that in my to do list , Will share you result tomorrow IST .
regards
@rvind

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Keywords: regression

(In reply to Valentin Gosu [:valentin] (he/him) from comment #14)

(In reply to Arvind Kumar patel from comment #13)

(In reply to Valentin Gosu [:valentin] (he/him) from comment #11)
I have already tried with 67.0.3 and its not working.

Thanks for checking. That's good to know.

Can you give me today time and I will respond with your comment-8 debug result. Also if you want I can give you dockerfile that would be a amin job for you to reproduce with any particular version.

If you can find the regression range that would be very helpful.
Otherwise please post the docker file and we'll try to reproduce it sometimes in the next week. Thanks!

Hello Valentin,
I found the breaking build version. I haven't use your toll as it was much easier for me to do in dockerfile. The breaking BUILD is in 65.0.

I have run test on below build and found it is working till 64.0.2 but not with 65.0.1

Here is the result of build:
LAST WORKING: 62.0.2, 64.0.2
NOT WORKING: 67.0.3, 67.0.4, 65.0.2, 65.0.1
RESULT for FAILURE BUILD: 65.0.1

BELOW is the logs of these build which might be helpful for debug the issue:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//////////////////////////////////////////////////////////////////
LOG-FIREFOX-VERSION-67.0.4 -FAIL
/////////////////////////////////////////////////

root@fba6c4baac49:/app# cat geckodriver.log
1561528381149 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "--headless" "--disable-gpu" "--no-sandbox" "window-size=1200,1100" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.IJm3DmyhW1Z2"
1561528381150 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:43463
*** You are running in headless mode.
1561528381659 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1561528381660 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1561528381660 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1561528381811 Marionette TRACE Received observer notification profile-after-change
1561528381849 Marionette TRACE Received observer notification command-line-startup
1561528381849 Marionette TRACE Received observer notification nsPref:changed
1561528381849 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false)
1561528381972 Marionette TRACE Received observer notification toplevel-window-ready
1561528383153 Marionette TRACE Received observer notification marionette-startup-requested
1561528383154 Marionette TRACE Waiting until startup recorder finished recording startup scripts...
1561528383203 Marionette TRACE All scripts recorded.
1561528383203 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000
1561528383203 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99
1561528383203 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true
1561528383204 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false
1561528383204 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1561528383205 Marionette DEBUG Setting recommended pref browser.search.update to false
1561528383205 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false
1561528383205 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1561528383206 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false
1561528383206 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to
1561528383206 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false
1561528383207 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1561528383207 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true
1561528383207 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false
1561528383207 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1561528383207 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0
1561528383207 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0
1561528383208 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false
1561528383208 Marionette DEBUG Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1561528383209 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false
1561528383209 Marionette DEBUG Setting recommended pref network.http.speculative-parallel-limit to 0
1561528383209 Marionette DEBUG Setting recommended pref security.certerrors.mitm.priming.enabled to false
1561528383209 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false
1561528383209 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0
1561528383209 Marionette DEBUG Setting recommended pref signon.autofillForms to false
1561528383210 Marionette DEBUG Setting recommended pref signon.rememberSignons to false
1561528383210 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1561528383259 Marionette INFO Listening on port 43463
1561528383259 Marionette DEBUG Remote service is active
1561528383355 geckodriver::marionette DEBUG Connection established on 127.0.0.1:43463. Waiting for Marionette handshake
1561528383357 Marionette DEBUG Accepted connection 0 from 127.0.0.1:53006
1561528383358 geckodriver::marionette DEBUG Connected to Marionette
1561528383360 Marionette DEBUG 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1561528383361 Marionette WARN TLS certificate errors will be ignored for this session
1561528386877 Marionette TRACE [4294967297] Frame script loaded
1561528397084 Marionette TRACE [4294967297] Frame script registered
1561528413715 Marionette DEBUG 0 <- [1,1,null,{"sessionId":"adf32810-5e12-4526-88cb-be14c4e33689","capabilities":{"browserName":"firefox","browserVersion":"67.0. ... ozprofile.IJm3DmyhW1Z2","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1561528413732 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"adf32810-5e12-4526-88cb-be14c4e33689","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"67.0.4","moz:accessibilityChecks":false,"moz:buildID":"20190619235627","moz:geckodriverVersion":"0.24.0","moz:headless":true,"moz:processID":1508,"moz:profile":"/tmp/rust_mozprofile.IJm3DmyhW1Z2","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.9.0-4-amd64","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
1561528413734 webdriver::server DEBUG -> POST /session/adf32810-5e12-4526-88cb-be14c4e33689/url {"url": "https://google.com"}
1561528413744 Marionette DEBUG 0 -> [0,2,"WebDriver:Navigate",{"url":"https://google.com"}]
1561528413763 Marionette TRACE [4294967297] Received DOM event beforeunload for about:blank
[Child 1661, Chrome_ChildThread] WARNING: pipe error (31): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 357
[Child 1730, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 357
Exiting due to channel error.
[Child 1661, Chrome_ChildThread] WARNING: pipe error (52): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 357
[Child 1633, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 357
Exiting due to channel error.
[Child 1661, Chrome_ChildThread] WARNING: pipe error: Broken pipe: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 726
Exiting due to channel error.
root@fba6c4baac49:/app#

/////////////////////////////////////////////////////////////////////////////////////////////
LOG - FIREFOX-VERSION- 64.0.2 -PASS
//////////////////////////////////////////////////////////////////////////////////////////////

1561528629773 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "--headless" "--disable-gpu" "--no-sandbox" "window-size=1200,1100" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.5lNOvEof6Sqq"
1561528629774 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:34341
*** You are running in headless mode.
1561528630179 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1561528630179 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
1561528630326 Marionette DEBUG Received observer notification profile-after-change
1561528630394 Marionette DEBUG Received observer notification command-line-startup
1561528630394 Marionette DEBUG Received observer notification nsPref:changed
1561528630394 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false)
1561528630503 Marionette DEBUG Received observer notification toplevel-window-ready
1561528631355 Marionette DEBUG Received observer notification sessionstore-windows-restored
1561528631355 Marionette DEBUG Waiting for delayed startup...
1561528631638 Marionette DEBUG Setting recommended pref security.turn_off_all_security_so_that_viruses_can_take_over_this_computer to true
1561528631638 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000
1561528631639 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99
1561528631639 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true
1561528631639 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false
1561528631641 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1561528631642 Marionette DEBUG Setting recommended pref browser.search.update to false
1561528631642 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false
1561528631642 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1561528631642 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false
1561528631643 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to
1561528631643 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false
1561528631644 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1561528631645 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true
1561528631645 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false
1561528631645 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1561528631645 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0
1561528631645 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0
1561528631646 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false
1561528631646 Marionette DEBUG Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1561528631647 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false
1561528631647 Marionette DEBUG Setting recommended pref network.http.speculative-parallel-limit to 0
1561528631647 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false
1561528631647 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0
1561528631647 Marionette DEBUG Setting recommended pref signon.autofillForms to false
1561528631647 Marionette DEBUG Setting recommended pref signon.rememberSignons to false
1561528631647 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1561528631693 Marionette INFO Listening on port 34341
1561528631694 Marionette DEBUG Remote service is active
1561528631779 geckodriver::marionette DEBUG Connection established on 127.0.0.1:34341. Waiting for Marionette handshake
1561528631780 Marionette DEBUG Accepted connection 0 from 127.0.0.1:59640
1561528631781 geckodriver::marionette DEBUG Connected to Marionette
1561528631783 Marionette TRACE 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1561528631785 Marionette WARN TLS certificate errors will be ignored for this session
1561528631825 Marionette DEBUG [4294967297] Frame script loaded
1561528631826 Marionette DEBUG [4294967297] Frame script registered
1561528631872 Marionette TRACE 0 <- [1,1,null,{"sessionId":"51bb2196-fc99-4326-8eab-01cf3df28f84","capabilities":{"browserName":"firefox","browserVersion":"64.0. ... ozprofile.5lNOvEof6Sqq","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1561528631874 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"51bb2196-fc99-4326-8eab-01cf3df28f84","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"64.0.2","moz:accessibilityChecks":false,"moz:geckodriverVersion":"0.24.0","moz:headless":true,"moz:processID":1791,"moz:profile":"/tmp/rust_mozprofile.5lNOvEof6Sqq","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.9.0-4-amd64","rotatable":false,"setWindowRect":true,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
1561528631877 webdriver::server DEBUG -> POST /session/51bb2196-fc99-4326-8eab-01cf3df28f84/url {"url": "https://google.com"}
1561528631879 Marionette TRACE 0 -> [0,2,"WebDriver:Navigate",{"url":"https://google.com"}]
1561528631893 Marionette DEBUG [4294967297] Received DOM event beforeunload for about:neterror?e=dnsNotFound&u=http%3A//www.window-size%3D1200%2C1100.com/&c=UTF-8&f=regular&d=We%20can%E2%80%99t%20connect%20to%20the%20server%20at%20www.window-size%3D1200%2C1100.com.
1561528632123 Marionette DEBUG [4294967297] Received DOM event pagehide for about:neterror?e=dnsNotFound&u=http%3A//www.window-size%3D1200%2C1100.com/&c=UTF-8&f=regular&d=We%20can%E2%80%99t%20connect%20to%20the%20server%20at%20www.window-size%3D1200%2C1100.com.
1561528632125 Marionette DEBUG [4294967297] Received DOM event unload for about:neterror?e=dnsNotFound&u=http%3A//www.window-size%3D1200%2C1100.com/&c=UTF-8&f=regular&d=We%20can%E2%80%99t%20connect%20to%20the%20server%20at%20www.window-size%3D1200%2C1100.com.
1561528632206 Marionette DEBUG [4294967297] Received DOM event DOMContentLoaded for https://www.google.com/
1561528632628 Marionette DEBUG [4294967297] Received DOM event pageshow for https://www.google.com/
1561528632630 Marionette TRACE 0 <- [1,2,null,{"value":null}]
1561528632633 webdriver::server DEBUG <- 200 OK {"value":null}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

		  LOG - FIREFOX-VERSION- 65.0.2 - FAIL

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

1561528976971 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "--headless" "--disable-gpu" "--no-sandbox" "window-size=1200,1100" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.THKuPsTkK94g"
1561528976972 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:39053
*** You are running in headless mode.
1561528977428 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1561528977428 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
1561528977591 Marionette TRACE Received observer notification profile-after-change
1561528977655 Marionette TRACE Received observer notification command-line-startup
1561528977655 Marionette TRACE Received observer notification nsPref:changed
1561528977655 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false)
1561528977776 Marionette TRACE Received observer notification toplevel-window-ready
1561528978788 Marionette TRACE Received observer notification sessionstore-windows-restored
1561528978789 Marionette TRACE Waiting until startup recorder finished recording startup scripts...
1561528978933 Marionette TRACE All scripts recorded.
1561528978933 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000
1561528978934 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99
1561528978934 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true
1561528978934 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false
1561528978934 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1561528978935 Marionette DEBUG Setting recommended pref browser.search.update to false
1561528978935 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false
1561528978935 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1561528978936 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false
1561528978936 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to
1561528978936 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false
1561528978937 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1561528978937 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true
1561528978937 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false
1561528978937 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1561528978937 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0
1561528978938 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0
1561528978938 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false
1561528978938 Marionette DEBUG Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1561528978939 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false
1561528978939 Marionette DEBUG Setting recommended pref network.http.speculative-parallel-limit to 0
1561528978939 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false
1561528978939 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0
1561528978939 Marionette DEBUG Setting recommended pref signon.autofillForms to false
1561528978940 Marionette DEBUG Setting recommended pref signon.rememberSignons to false
1561528978940 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1561528978999 Marionette INFO Listening on port 39053
1561528978999 Marionette DEBUG Remote service is active
1561528979077 geckodriver::marionette DEBUG Connection established on 127.0.0.1:39053. Waiting for Marionette handshake
1561528979079 Marionette DEBUG Accepted connection 0 from 127.0.0.1:41654
1561528979081 geckodriver::marionette DEBUG Connected to Marionette
1561528979084 Marionette DEBUG 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1561528979085 Marionette WARN TLS certificate errors will be ignored for this session
1561528990849 Marionette TRACE [4294967297] Frame script loaded
1561529007700 Marionette TRACE [4294967297] Frame script registered
1561529023366 Marionette DEBUG 0 <- [1,1,null,{"sessionId":"22662f85-993f-4c01-a3d8-d377e4103451","capabilities":{"browserName":"firefox","browserVersion":"65.0. ... ozprofile.THKuPsTkK94g","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1561529023406 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"22662f85-993f-4c01-a3d8-d377e4103451","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"65.0.2","moz:accessibilityChecks":false,"moz:geckodriverVersion":"0.24.0","moz:headless":true,"moz:processID":2635,"moz:profile":"/tmp/rust_mozprofile.THKuPsTkK94g","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.9.0-4-amd64","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
1561529023410 webdriver::server DEBUG -> POST /session/22662f85-993f-4c01-a3d8-d377e4103451/url {"url": "https://google.com"}
1561529023417 Marionette DEBUG 0 -> [0,2,"WebDriver:Navigate",{"url":"https://google.com"}]
[Child 2785, Chrome_ChildThread] WARNING: pipe error (53): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349
[Child 2753, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349
[Child 2785, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349

///////////////////////////////////////////////////////////////
LOG - FIREFOX-VERSION- 65.0.1 - FAIL
/////////////////////////////////////////////////////////

1561529227698 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "--headless" "--disable-gpu" "--no-sandbox" "window-size=1200,1100" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.lD4Zbys8yLGf"
1561529227699 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:36115
*** You are running in headless mode.
1561529228137 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1561529228137 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
1561529228286 Marionette TRACE Received observer notification profile-after-change
1561529228353 Marionette TRACE Received observer notification command-line-startup
1561529228354 Marionette TRACE Received observer notification nsPref:changed
1561529228354 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false)
1561529228463 Marionette TRACE Received observer notification toplevel-window-ready
1561529229380 Marionette TRACE Received observer notification sessionstore-windows-restored
1561529229381 Marionette TRACE Waiting until startup recorder finished recording startup scripts...
1561529229520 Marionette TRACE All scripts recorded.
1561529229520 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000
1561529229520 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99
1561529229521 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true
1561529229521 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false
1561529229521 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1561529229523 Marionette DEBUG Setting recommended pref browser.search.update to false
1561529229524 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false
1561529229524 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1561529229524 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false
1561529229525 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to
1561529229525 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false
1561529229527 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1561529229527 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true
1561529229527 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false
1561529229528 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1561529229528 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0
1561529229528 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0
1561529229529 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false
1561529229529 Marionette DEBUG Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1561529229530 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false
1561529229530 Marionette DEBUG Setting recommended pref network.http.speculative-parallel-limit to 0
1561529229530 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false
1561529229530 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0
1561529229531 Marionette DEBUG Setting recommended pref signon.autofillForms to false
1561529229531 Marionette DEBUG Setting recommended pref signon.rememberSignons to false
1561529229531 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1561529229598 Marionette INFO Listening on port 36115
1561529229598 Marionette DEBUG Remote service is active
1561529229604 geckodriver::marionette DEBUG Connection established on 127.0.0.1:36115. Waiting for Marionette handshake
1561529229619 Marionette DEBUG Accepted connection 0 from 127.0.0.1:54392
1561529229627 geckodriver::marionette DEBUG Connected to Marionette
1561529229639 Marionette DEBUG 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1561529229640 Marionette WARN TLS certificate errors will be ignored for this session
1561529229935 Marionette TRACE [4294967297] Frame script loaded
1561529230502 Marionette TRACE [4294967297] Frame script registered
1561529240318 Marionette DEBUG 0 <- [1,1,null,{"sessionId":"b79e514b-6a11-411c-9f45-c4af1c201a6e","capabilities":{"browserName":"firefox","browserVersion":"65.0. ... ozprofile.lD4Zbys8yLGf","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1561529240330 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"b79e514b-6a11-411c-9f45-c4af1c201a6e","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"65.0.1","moz:accessibilityChecks":false,"moz:geckodriverVersion":"0.24.0","moz:headless":true,"moz:processID":2896,"moz:profile":"/tmp/rust_mozprofile.lD4Zbys8yLGf","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.9.0-4-amd64","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
1561529240334 webdriver::server DEBUG -> POST /session/b79e514b-6a11-411c-9f45-c4af1c201a6e/url {"url": "https://google.com"}
1561529240339 Marionette DEBUG 0 -> [0,2,"WebDriver:Navigate",{"url":"https://google.com"}]
1561529240355 Marionette TRACE [4294967297] Received DOM event beforeunload for about:blank
[Child 3101, Chrome_ChildThread] WARNING: pipe error (28): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349
Exiting due to channel error.
[Child 3048, Chrome_ChildThread] WARNING: pipe error (27): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349
[Child 3048, Chrome_ChildThread] WARNING: pipe error (52): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349
[Child 3016, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349
[Child 3048, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 349

///////////////////////////

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Has Regression Range: --- → yes

(In reply to Arvind Kumar patel from comment #17)

(In reply to Valentin Gosu [:valentin] (he/him) from comment #14)

(In reply to Arvind Kumar patel from comment #13)

(In reply to Valentin Gosu [:valentin] (he/him) from comment #11)
I have already tried with 67.0.3 and its not working.

Thanks for checking. That's good to know.

Can you give me today time and I will respond with your comment-8 debug result. Also if you want I can give you dockerfile that would be a amin job for you to reproduce with any particular version.

If you can find the regression range that would be very helpful.
Otherwise please post the docker file and we'll try to reproduce it sometimes in the next week. Thanks!

Hello Valentin,
I found the breaking build version. I haven't use your toll as it was much easier for me to do in dockerfile. The breaking BUILD is in 65.0.

Thanks Arvind. I still wish you used mozregression, if possible, or share the dockerfile with us.
mozregression is able to identify the push that caused the regression, instead of just the build when it started happening.
That way we don't have to guess which one of the thousands of commits caused the problem to occur.

I have run test on below build and found it is working till 64.0.2 but not with 65.0.1

Here is the result of build:
LAST WORKING: 62.0.2, 64.0.2
NOT WORKING: 67.0.3, 67.0.4, 65.0.2, 65.0.1
RESULT for FAILURE BUILD: 65.0.1

This is a good starting point, thanks.
Here is an example of using mozregression with a custom command:
https://mozilla.github.io/mozregression/documentation/automatic-bisection.html

Flags: needinfo?(arvind1843)
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.