Closed Bug 1316665 Opened 3 years ago Closed 8 months ago

Never ending spinners on a freshly installed Nightly 64-bit

Categories

(Core :: Security: Process Sandboxing, defect, P3)

52 Branch
x86_64
Windows 7
defect

Tracking

()

RESOLVED INVALID
Tracking Status
firefox52 --- wontfix
firefox55 --- affected
firefox56 --- affected
firefox57 --- affected
firefox58 --- affected
firefox59 --- ?

People

(Reporter: alex_mayorga, Unassigned)

Details

(Keywords: nightly-community, Whiteboard: sb+)

Attachments

(2 files)

¡Hola!

So I might or might not have tried to install Nightly in some shiny "virtualized environment".

The install went fine and I did import settings from IE during the install.

But once I clicked run I was greeted by two tabs with never ending spinners =(

Below is what the "Browser Console" had.

¡Gracias!
Alex


NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]  XULStore.js:66
PAC file installed from http://webproxy.somewhere.net:8080/
Win error 2 during operation open on file C:\Users\me\AppData\Local\Mozilla\Firefox\Profiles\vbkacf3w.default\directoryLinks.json (The system cannot find the file specified.

)  DirectoryLinksProvider.jsm:402
	DirectoryLinksProvider_readDirectoryLinksFile/< resource://app/modules/DirectoryLinksProvider.jsm:402:7
	Handler.prototype.process resource://gre/modules/Promise-backend.js:935:21
	this.PromiseWalker.walkerLoop resource://gre/modules/Promise-backend.js:813:7
	bound  self-hosted
	bound bound  self-hosted
Key event not available on some keyboard layouts: key=“r” modifiers=“accel,alt” id=“toggleReaderMode”  browser.xul
Key event not available on some keyboard layouts: key=“i” modifiers=“accel,alt,shift” id=“key_browserToolbox”  browser.xul
1478803605440	addons.update-checker	WARN	Request failed: https://versioncheck-bg.addons.mozilla.org/update/VersionCheck.php?reqVersion=2&id=aushelper@mozilla.org&version=1.0&maxAppVersion=52.0a1&status=userEnabled&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=52.0a1&appOS=WINNT&appABI=x86_64-msvc&locale=en-US&currentAppVersion=52.0a1&updateType=112&compatMode=normal - 502: Bad Gateway
1478803605788	addons.update-checker	WARN	Update manifest for formautofill@mozilla.org did not contain an updates property
1478803605817	addons.update-checker	WARN	Update manifest for flyweb@mozilla.org did not contain an updates property
1478803606004	addons.update-checker	WARN	Update manifest for webcompat@mozilla.org did not contain an updates property
1478803606018	addons.update-checker	WARN	Update manifest for firefox@getpocket.com did not contain an updates property
1478803606082	addons.update-checker	WARN	Update manifest for {972ce4c6-7e08-4474-a285-3208198ce6fd} did not contain an updates property
1478803607281	addons.update-checker	WARN	Update manifest for e10srollout@mozilla.org did not contain an updates property
Component: General → Untriaged
Hey alex_mayorga, are you able to reproduce reliably? If so, is the problem reproducible if layers.gpu-process.dev.enabled is set to false? If so, this is perhaps a dupe of bug 1316632.
Flags: needinfo?(alex_mayorga)
¡Hola Mike!

layers.gpu-process.dev.enabled set to false did not have an effect.

I've tried setting Nightly to "Auto-detect proxy settings for this network" but it still doesn't work.

Looks like the WPAD for this network is broken a bit and it throws off Nightly completely.

Below is what the "Browser Console" has now.

Any other ideas as to what to try in this scenario?

¡Gracias!
Alex

1479743974204	Browser.Experiments.Experiments	ERROR	Experiments #0::httpGetRequest::onError() - Error making request to https://telemetry-experiment.cdn.mozilla.net/manifest/v1/firefox/53.0a1/nightly: error  Log.jsm:753
1479743974205	Browser.Experiments.Experiments	ERROR	Experiments #0::_loadManifest - failure to fetch/parse manifest (continuing anyway): Error: Experiments - XHR error for https://telemetry-experiment.cdn.mozilla.net/manifest/v1/firefox/53.0a1/nightly - error  Log.jsm:753
1479743974324	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 0).
Key event not available on some keyboard layouts: key=“r” modifiers=“accel,alt” id=“toggleReaderMode”  browser.xul
Key event not available on some keyboard layouts: key=“i” modifiers=“accel,alt,shift” id=“key_browserToolbox”  browser.xul
PAC file failed to install from http://wpad/notify-Splash_Coach?aHR0cDovL3dwYWQvd3BhZC5kYXQ=
PAC Execution Error: expected expression, got '<' [<html>
]
1479743992615	Toolkit.Telemetry	ERROR	TelemetrySend::_doPing - error making request to https://incoming.telemetry.mozilla.org/submit/telemetry/9813d579-1130-4972-b8c4-00e66b9df48e/saved-session/Firefox/53.0a1/nightly/20161121030224?v=4: error  Log.jsm:753
1479743992616	Toolkit.Telemetry	ERROR	TelemetrySend::sendPersistedPings - failed to send ping 9813d579-1130-4972-b8c4-00e66b9df48e: {"isTrusted":true}  Log.jsm:753
1479743992616	Toolkit.Telemetry	ERROR	TelemetrySend::_doPing - error making request to https://incoming.telemetry.mozilla.org/submit/telemetry/67539f9c-6d60-4b63-bd1f-62606a2586d1/main/Firefox/53.0a1/nightly/20161121030224?v=4: error  Log.jsm:753
1479743992617	Toolkit.Telemetry	ERROR	TelemetrySend::sendPersistedPings - failed to send ping 67539f9c-6d60-4b63-bd1f-62606a2586d1: {"isTrusted":true}  Log.jsm:753
A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Mon Nov 21 2016 10:59:55 GMT-0500 (Eastern Standard Time)
Full Message: Error: page-thumbnail:error
Full Stack: observe@resource://gre/modules/BackgroundPageThumbs.jsm:116:22
BackgroundPageThumbs._onCaptureOrTimeout@resource://gre/modules/BackgroundPageThumbs.jsm:305:7
Capture.prototype._done/done@resource://gre/modules/BackgroundPageThumbs.jsm:451:7
Capture.prototype._done@resource://gre/modules/BackgroundPageThumbs.jsm:471:7
Capture.prototype.notify@resource://gre/modules/BackgroundPageThumbs.jsm:428:5
  BackgroundPageThumbs.jsm:116
1479744004337	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 1).
A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Mon Nov 21 2016 11:00:25 GMT-0500 (Eastern Standard Time)
Full Message: Error: page-thumbnail:error
Full Stack: observe@resource://gre/modules/BackgroundPageThumbs.jsm:116:22
BackgroundPageThumbs._onCaptureOrTimeout@resource://gre/modules/BackgroundPageThumbs.jsm:305:7
Capture.prototype._done/done@resource://gre/modules/BackgroundPageThumbs.jsm:451:7
Capture.prototype._done@resource://gre/modules/BackgroundPageThumbs.jsm:471:7
Capture.prototype.notify@resource://gre/modules/BackgroundPageThumbs.jsm:428:5
  BackgroundPageThumbs.jsm:116
1479744064316	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 2).
A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Mon Nov 21 2016 11:00:55 GMT-0500 (Eastern Standard Time)
Full Message: Error: page-thumbnail:error
Full Stack: observe@resource://gre/modules/BackgroundPageThumbs.jsm:116:22
BackgroundPageThumbs._onCaptureOrTimeout@resource://gre/modules/BackgroundPageThumbs.jsm:305:7
Capture.prototype._done/done@resource://gre/modules/BackgroundPageThumbs.jsm:451:7
Capture.prototype._done@resource://gre/modules/BackgroundPageThumbs.jsm:471:7
Capture.prototype.notify@resource://gre/modules/BackgroundPageThumbs.jsm:428:5
  BackgroundPageThumbs.jsm:116
1479744082578	addons.repository	WARN	Search failed when repopulating cache
1479744082904	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082906	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082907	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082909	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082910	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082911	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082914	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082915	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744082933	addons.update-checker	WARN	HTTP Request failed for an unknown reason
1479744083197	addons.productaddons	WARN	Failed downloading XML, status: 0, reason: error
1479744083230	addons.productaddons	WARN	Failed downloading via XHR, status: 0, reason: error
1479744083230	addons.xpi	ERROR	Failed to download system add-on gmp-widevinecdm: Error: Failed downloading via XHR, status: 0, reason: error (resource://gre/modules/addons/ProductAddonChecker.jsm:316:16) JS Stack trace: downloadFile/</fail@ProductAddonChecker.jsm:316:16 < EventListener.handleEvent*downloadFile/<@ProductAddonChecker.jsm:320:5 < downloadFile@ProductAddonChecker.jsm:292:10 < ProductAddonChecker.downloadAddon<@ProductAddonChecker.jsm:438:22  Log.jsm:753
1479744083361	addons.productaddons	WARN	Failed downloading via XHR, status: 0, reason: error
1479744083361	addons.xpi	ERROR	Failed to download system add-on gmp-eme-adobe: Error: Failed downloading via XHR, status: 0, reason: error (resource://gre/modules/addons/ProductAddonChecker.jsm:316:16) JS Stack trace: downloadFile/</fail@ProductAddonChecker.jsm:316:16 < EventListener.handleEvent*downloadFile/<@ProductAddonChecker.jsm:320:5 < downloadFile@ProductAddonChecker.jsm:292:10 < ProductAddonChecker.downloadAddon<@ProductAddonChecker.jsm:438:22  Log.jsm:753
1479744083476	addons.productaddons	WARN	Failed downloading via XHR, status: 0, reason: error
1479744083477	addons.xpi	ERROR	Failed to download system add-on gmp-gmpopenh264: Error: Failed downloading via XHR, status: 0, reason: error (resource://gre/modules/addons/ProductAddonChecker.jsm:316:16) JS Stack trace: downloadFile/</fail@ProductAddonChecker.jsm:316:16 < EventListener.handleEvent*downloadFile/<@ProductAddonChecker.jsm:320:5 < downloadFile@ProductAddonChecker.jsm:292:10 < ProductAddonChecker.downloadAddon<@ProductAddonChecker.jsm:438:22  Log.jsm:753
1479744083479	addons.manager	WARN	Failed to update system addons: Error: Rejecting updated system add-on set that either could not be downloaded or contained unusable add-ons. (resource://gre/modules/addons/XPIProvider.jsm:3223:13) JS Stack trace: this.XPIProvider.updateSystemAddons<@XPIProvider.jsm:3223:13
1479744112642	Toolkit.Telemetry	ERROR	TelemetrySend::_doPing - error making request to https://incoming.telemetry.mozilla.org/submit/telemetry/9813d579-1130-4972-b8c4-00e66b9df48e/saved-session/Firefox/53.0a1/nightly/20161121030224?v=4: error  Log.jsm:753
1479744112642	Toolkit.Telemetry	ERROR	TelemetrySend::sendPersistedPings - failed to send ping 9813d579-1130-4972-b8c4-00e66b9df48e: {"isTrusted":true}  Log.jsm:753
1479744112643	Toolkit.Telemetry	ERROR	TelemetrySend::_doPing - error making request to https://incoming.telemetry.mozilla.org/submit/telemetry/67539f9c-6d60-4b63-bd1f-62606a2586d1/main/Firefox/53.0a1/nightly/20161121030224?v=4: error  Log.jsm:753
1479744112644	Toolkit.Telemetry	ERROR	TelemetrySend::sendPersistedPings - failed to send ping 67539f9c-6d60-4b63-bd1f-62606a2586d1: {"isTrusted":true}  Log.jsm:753
A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Mon Nov 21 2016 11:01:25 GMT-0500 (Eastern Standard Time)
Full Message: Error: page-thumbnail:error
Full Stack: observe@resource://gre/modules/BackgroundPageThumbs.jsm:116:22
BackgroundPageThumbs._onCaptureOrTimeout@resource://gre/modules/BackgroundPageThumbs.jsm:305:7
Capture.prototype._done/done@resource://gre/modules/BackgroundPageThumbs.jsm:451:7
Capture.prototype._done@resource://gre/modules/BackgroundPageThumbs.jsm:471:7
Capture.prototype.notify@resource://gre/modules/BackgroundPageThumbs.jsm:428:5
  BackgroundPageThumbs.jsm:116
A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Mon Nov 21 2016 11:01:55 GMT-0500 (Eastern Standard Time)
Full Message: Error: page-thumbnail:error
Full Stack: observe@resource://gre/modules/BackgroundPageThumbs.jsm:116:22
BackgroundPageThumbs._onCaptureOrTimeout@resource://gre/modules/BackgroundPageThumbs.jsm:305:7
Capture.prototype._done/done@resource://gre/modules/BackgroundPageThumbs.jsm:451:7
Capture.prototype._done@resource://gre/modules/BackgroundPageThumbs.jsm:471:7
Capture.prototype.notify@resource://gre/modules/BackgroundPageThumbs.jsm:428:5
  BackgroundPageThumbs.jsm:116
1479744154297	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 3).
1479744274278	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 4).
1479744352712	Toolkit.Telemetry	ERROR	TelemetrySend::_doPing - error making request to https://incoming.telemetry.mozilla.org/submit/telemetry/9813d579-1130-4972-b8c4-00e66b9df48e/saved-session/Firefox/53.0a1/nightly/20161121030224?v=4: error  Log.jsm:753
1479744352712	Toolkit.Telemetry	ERROR	TelemetrySend::sendPersistedPings - failed to send ping 9813d579-1130-4972-b8c4-00e66b9df48e: {"isTrusted":true}  Log.jsm:753
1479744352714	Toolkit.Telemetry	ERROR	TelemetrySend::_doPing - error making request to https://incoming.telemetry.mozilla.org/submit/telemetry/67539f9c-6d60-4b63-bd1f-62606a2586d1/main/Firefox/53.0a1/nightly/20161121030224?v=4: error  Log.jsm:753
1479744352714	Toolkit.Telemetry	ERROR	TelemetrySend::sendPersistedPings - failed to send ping 67539f9c-6d60-4b63-bd1f-62606a2586d1: {"isTrusted":true}  Log.jsm:753
1479744424287	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 5).
XML Parsing Error: no root element found
Location: jar:file:///C:/Users/user/AppData/Local/Nightly/browser/omni.ja!/chrome/browser/content/browser/aboutNetError.xhtml
Line Number 1, Column 1:  aboutNetError.xhtml:1:1
Flags: needinfo?(alex_mayorga) → needinfo?(mconley)
Hey alex_mayorga. This is conflict with your comment in bug 1316632 comment 18. Is that comment no longer accurate? What changed?

Have you had any success finding a regression range? Do you know how I can configure a WPAD to reproduce this behaviour locally?
Flags: needinfo?(mconley) → needinfo?(alex_mayorga)
Also, perhaps I'm not understanding what you mean by "never ending spinners". Do you mean tab switch spinners in the center of the content area, or tab throbbers that indicate loading?
¡Hola Mike!

My comment in bug 1316632 comment 18 is from a different physical laptop.

I have yet to setup mozregression here, but Developer works.

I unfortunately don't know anything about WPAD.

The spinners are in the center of the content area.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga)
I'm at a loss here with regards to how I reproduce this. Is there a public DNS server I can point at that's serving the misconfigured WPAD?

Are you able to reproduce with add-ons disabled?
Flags: needinfo?(alex_mayorga)
¡Hola Mike!

This is a completely new install, so there are no add-ons and the profile is new.

There's a bug in mozregression where it doesn't try to install to AppLocal if Program Files is not writable that is the case for this system.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga)
Unfortunately, I still don't have enough STR / know-how to reproduce this, and without reproduction, I don't have a great chance of diagnosing it.

Hey RyanVM, would it be possible for someone from SV to work with alex_mayorga to develop some STR for me?
Flags: needinfo?(ryanvm)
Flags: needinfo?(ryanvm) → needinfo?(rares.bologa)
Flags: needinfo?(rares.bologa) → needinfo?(roxana.leitan)
¡Hola!

Testing with Roxana via IRC.

This WFM on 32-bit Nightly it seems.

Dropping about:buildconfig below as a data point.

Will test again on 64-bit Nightly and confirm.

¡Gracias!
Alex

about:buildconfig
Source

Built from https://hg.mozilla.org/mozilla-central/rev/0534254e9a40b4bade2577c631fe4cfa0b5db41d
Build platform
target
i686-pc-mingw32
Build tools
Compiler 	Version 	Compiler flags
c:/builds/moz2_slave/m-cen-w32-ntly-000000000000000/build/src/vs2015u3/VC/bin/amd64_x86/cl.EXE 	19.00.24213 	-TC -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -utf-8 -arch:SSE2 -FS -Gw -wd4244 -wd4267 -we4553
c:/builds/moz2_slave/m-cen-w32-ntly-000000000000000/build/src/vs2015u3/VC/bin/amd64_x86/cl.EXE 	19.00.24213 	-TP -nologo -wd5026 -wd5027 -Zc:sizedDealloc- -Zc:threadSafeInit- -wd4091 -wd4577 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -utf-8 -arch:SSE2 -FS -Gw -wd4251 -wd4244 -wd4267 -wd4345 -wd4351 -wd4800 -wd4595 -we4553 -GR- -Zi -GL -wd4624 -wd4952 -O1 -Oi -Oy-
Configure options

MOZ_AUTOMATION=1 'MOZILLABUILD=C:\mozilla-build' --enable-update-channel=nightly MOZ_PGO=1 WINDOWSSDKDIR=c:/builds/moz2_slave/m-cen-w32-ntly-000000000000000/build/src/vs2015u3/SDK --enable-rust RUSTC=c:/builds/moz2_slave/m-cen-w32-ntly-000000000000000/build/src/rustc/bin/rustc CARGO=c:/builds/moz2_slave/m-cen-w32-ntly-000000000000000/build/src/rustc/bin/cargo --enable-jemalloc --enable-js-shell --enable-profiling --enable-eme=+adobe --with-mozilla-api-keyfile=/c/builds/mozilla-desktop-geoloc-api.key --with-google-api-keyfile=/c/builds/gapi.data MAKE=c:/builds/moz2_slave/m-cen-w32-ntly-000000000000000/build/src/mozmake.EXE --enable-crashreporter --enable-release --enable-require-all-d3dc-versions --enable-verify-mar --with-branding=browser/branding/nightly
¡Hola!

This seems to be a 32-bit vs. 64-bit bug.

Attached a screen capture of the issue.

Below the about:buildconfig of the problematic one.

¡Gracias!
Alex

about:buildconfig
Source

Built from https://hg.mozilla.org/mozilla-central/rev/0534254e9a40b4bade2577c631fe4cfa0b5db41d
Build platform
target
x86_64-pc-mingw32
Build tools
Compiler 	Version 	Compiler flags
c:/builds/moz2_slave/m-cen-w64-ntly-000000000000000/build/src/vs2015u3/VC/bin/amd64/cl.EXE 	19.00.24213 	-TC -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -utf-8 -FS -Gw -wd4244 -wd4267 -we4553
c:/builds/moz2_slave/m-cen-w64-ntly-000000000000000/build/src/vs2015u3/VC/bin/amd64/cl.EXE 	19.00.24213 	-TP -nologo -wd5026 -wd5027 -Zc:sizedDealloc- -Zc:threadSafeInit- -wd4091 -wd4577 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -utf-8 -FS -Gw -wd4251 -wd4244 -wd4267 -wd4345 -wd4351 -wd4800 -wd4595 -we4553 -GR- -Zi -GL -wd4624 -wd4952 -O1 -Oi -Oy-
Configure options

MOZ_AUTOMATION=1 'MOZILLABUILD=C:\mozilla-build' --host=x86_64-pc-mingw32 --target=x86_64-pc-mingw32 --enable-update-channel=nightly MOZ_PGO=1 WINDOWSSDKDIR=c:/builds/moz2_slave/m-cen-w64-ntly-000000000000000/build/src/vs2015u3/SDK --enable-rust RUSTC=c:/builds/moz2_slave/m-cen-w64-ntly-000000000000000/build/src/rustc/bin/rustc CARGO=c:/builds/moz2_slave/m-cen-w64-ntly-000000000000000/build/src/rustc/bin/cargo --enable-jemalloc --enable-js-shell --enable-profiling --enable-eme=+adobe --with-mozilla-api-keyfile=/c/builds/mozilla-desktop-geoloc-api.key --with-google-api-keyfile=/c/builds/gapi.data MAKE=c:/builds/moz2_slave/m-cen-w64-ntly-000000000000000/build/src/mozmake.EXE --enable-crashreporter --enable-release --enable-verify-mar --with-branding=browser/branding/nightly
¡Hola Mike!

Below the contents of "Browser Console" at the time the spinners are seen.

Anything else that we could try collecting?

¡Gracias!
Alex

PAC file installed from http://webproxy.somewhere.net:8080/
Key event not available on some keyboard layouts: key=“r” modifiers=“accel,alt” id=“toggleReaderMode”  browser.xul
Key event not available on some keyboard layouts: key=“i” modifiers=“accel,alt,shift” id=“key_browserToolbox”  browser.xul
Flags: needinfo?(mconley)
Summary: Never ending spinners on a freshly installed Nightly → Never ending spinners on a freshly installed Nightly 64-bit
Unfortunately, neither of those Browser Console log entries are very useful here.

You could try setting browser.tabs.remote.logSwitchTiming to true and collecting a log after that.

Would you be able to capture a short video demonstrating the bug?
Flags: needinfo?(mconley) → needinfo?(alex_mayorga)
Above all, a regression range would be very useful here. Or, information on whether or not e10s has _ever_ worked in this VM configuration.
¡Hola Mike!

We found out that a non e10s window works flawlessly on the problematic system.

How do I go about finding out what the exact bug in e10s is at play here?

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga) → needinfo?(mconley)
(In reply to alex_mayorga from comment #15)
> ¡Hola Mike!
> 
> We found out that a non e10s window works flawlessly on the problematic
> system.
> 
> How do I go about finding out what the exact bug in e10s is at play here?
> 
> ¡Gracias!
> Alex

Certainly a regression range would help if one could be found. That'd be hugely helpful.

Alternatively, would you be willing and able to bring this machine to Hawaii?
Flags: needinfo?(mconley) → needinfo?(alex_mayorga)
¡Hola Mike!

To hunt down a regression, I'd need https://bugzilla.mozilla.org/show_bug.cgi?id=1192488 fixed.

The laptop would most likely be going to Hawaii.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga)
(In reply to alex_mayorga from comment #17)
> ¡Hola Mike!
> 
> To hunt down a regression, I'd need
> https://bugzilla.mozilla.org/show_bug.cgi?id=1192488 fixed.
> 

Would you at the very least be able to determine if this is a new regression? Could you, perhaps, download a Nightly from earlier this year (perhaps July?) and see if you can reproduce to maybe give us a starting point on a regression window?
Flags: needinfo?(alex_mayorga)
¡Hola Mike!

Tried the build from July 1, 2016.

Still no go.

¡Gracias!
Alex

Could not read chrome manifest 'file:///C:/Users/me/AppData/Local/Nightly/chrome.manifest'.
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]XULStore.js:66
some cookies did not successfully migrate.
PAC file installed from http://webproxy.somewhere.net:8080/
Win error 2 during operation open on file C:\Users\am74839\AppData\Local\Mozilla\Firefox\Profiles\26ciqi3d.default\directoryLinks.json (The system cannot find the file specified.

)
DirectoryLinksProvider.jsm:402
self-repair.mozilla.org:443 usa un certificado de seguridad no válido.

No se confía en el certificado porque el certificado emisor es desconocido.
El servidor podría no estar enviando los certificados intermedios apropiados.
Puede ser necesario importar un certificado raíz adicional.

Error de código: <a id="errorCode" title="SEC_ERROR_UNKNOWN_ISSUER">SEC_ERROR_UNKNOWN_ISSUER</a>
(desconocido)
1480094412367	Browser.SelfSupportBackend	WARN	onLocationChange - There was a problem fetching the SelfSupport URL (attempt 0).
Flags: needinfo?(alex_mayorga)
Does the spinner show up if you have a traditional (non-WPAD) network configuration set up? I'm wondering if your "virtualized environment" that you spoke about in comment 0 is more at play here than the network configuration.
Flags: needinfo?(alex_mayorga)
Flags: needinfo?(roxana.leitan)
This is mconley writing from alex_mayorga's account in MozHawaii.

We've determined the following:

1) No, the problem is not currently fixed.
2) This is related to the sandbox. Setting security.sandbox.content.level to 0 addresses the issue. This explains why non-e10s windows continue to work, since non-e10s windows are not sandboxed.

Conditions that we're in:

We have a virtual machine running Windows 7, where the user account has no write access to Program Files, so Firefox (Nightly) has been installed to AppData. We notice that the content process is just not starting (according to bobowen, this is possibly due to the DLLs not loading).

bobowen is sitting to my right. He is going to look into this, and perhaps give a build to alex_mayorga to test.
Flags: needinfo?(alex_mayorga) → needinfo?(bobowencode)
I've pushed another build to try in an attempt to get some more logging out [1].
This build will just log all child file access issues to stderr.
However, it's possible that it is still failing too early even for this logging to work.

If you run from a command line, you should see the output in the terminal.
Alternatively you can add the following to the end of the command to run firefox, to get the output in a file (replace logfile.txt with whatever path you wish):

  2> logfile.txt

I've also made a small change to the sandbox policy, which might help.


[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=1a7268b7cf158caa086f6d756ad74624a2ce895d&selectedJob=32521069
Flags: needinfo?(bobowencode) → needinfo?(alex_mayorga)
Hi Alex,

Looks like I've got a fix for the similar issue in bug 1321256.
Hopefully it will solve this problem as well.

Can you try out the builds in this try push (the two different sorts of x86 and x64 builds should be the same):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=970d7aecb75b8ff18a1e2849e9a7401849f203a2

You will probably need to set the following pref to 1 (2 is the default for Nightly which would still fail.):
security.sandbox.content.level

You should either be able to do this in about:config by opening a new tab.
Or set it in your profile's pref.js file by adding the following:

user_pref("security.sandbox.content.level", 1);

Thanks
Component: Untriaged → Security: Process Sandboxing
Product: Firefox → Core
Whiteboard: sb?
¡Hola Bob!

I tried with https://archive.mozilla.org/pub/firefox/try-builds/bobowencode@gmail.com-970d7aecb75b8ff18a1e2849e9a7401849f203a2/try-win64/firefox-53.0a1.en-US.win64.installer.exe

security.sandbox.content.level to 0 Nightly does load page and this is what is logged:

[Child 21872] WARNING: pipe error: 109: file c:/builds/moz2_slave/try-w64-0000000000000000000000/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346

###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

[Child 31720] WARNING: pipe error: 232: file c:/builds/moz2_slave/try-w64-0000000000000000000000/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 513

###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost


###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost


###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost


###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost


###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost


###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost


###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

[Child 31720] WARNING: pipe error: 109: file c:/builds/moz2_slave/try-w64-0000000000000000000000/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346

security.sandbox.content.level to 1 Nightly does not load page, a "Nightly has stopped working" window pops up twice and nothing is logged.

security.sandbox.content.level to 2 Nightly does not load page, the never ending spinners are seen and nothing is logged.

Please let me know if there's anything else to try.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga) → needinfo?(bobowencode)
I was hoping that the same fix would work, but no such luck.

I've tried installing on a non-admin user on a VM and it installs to AppData\Local in the same way, but it works for me. Even if I remote desktop into the VM.
Unless I can reproduce, this is going to be hard to track down.
Only admin permissions (and a couple of other things) are removed from the content process token, so it should still have access to the user's own files.

Can you send me a screen print of the permissions on the Nightly folder that has just been installed into AppData\Local.

* Right click folder, select Properties
* Select Security tab
* Advanced button

Thanks.
Flags: needinfo?(bobowencode) → needinfo?(alex_mayorga)
Attached image Permissions
¡Hola Bob!

Attached please find the requested screen capture.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga)
Thanks Alex, unfortunately that all looks totally normal, so I'm at a bit of a loss as to what to try next.
(In reply to alex_mayorga from comment #27)
> Created attachment 8818310 [details]
> Permissions
> 
> ¡Hola Bob!
> 
> Attached please find the requested screen capture.
> 
> ¡Gracias!
> Alex

Would it be somehow possible to get bobowen a copy of the VM to tinker with?
Flags: needinfo?(alex_mayorga)
¡Hola Mike!

Sadly no, the regular corporate lingo and yada-yada applies here.

Perhaps some fail safes could be added into the code to disable e10s or sand-boxing "automagically" in this kind of scenario?

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga)
(In reply to alex_mayorga from comment #30)
> ¡Hola Mike!
> 
> Sadly no, the regular corporate lingo and yada-yada applies here.
> 
> Perhaps some fail safes could be added into the code to disable e10s or
> sand-boxing "automagically" in this kind of scenario?
> 
> ¡Gracias!
> Alex

Hm. I feel like if we could detect the conditions that cause this bug, we would be most of the way to understanding how to fixing it properly, as opposed to disabling the sandbox.

bobowen - suppose you had the computer sitting next to you at your desk. How would you approach solving this problem?
Flags: needinfo?(bobowencode)
Since we cannot give you a copy of the machine, I guess what's next is trying to find a way of re-creating the conditions of the machine. alex_mayorga: can you give us the following please?

1) The name and version of the virtualization software you're using
2) The version of Windows 7 installed on the machine
3) A sense of the access policies applied to the user account you were logged into in Hawaii.

It's possible that the group that assembled the VM has all of this information somewhere that allows them to easily compile and modify these VMs for distribution. Would it be possible to connect us with one of them?
Flags: needinfo?(alex_mayorga)
(In reply to Mike Conley (:mconley) from comment #31)
> (In reply to alex_mayorga from comment #30)
> > ¡Hola Mike!
> > 
> > Sadly no, the regular corporate lingo and yada-yada applies here.
> > 
> > Perhaps some fail safes could be added into the code to disable e10s or
> > sand-boxing "automagically" in this kind of scenario?
> > 
> > ¡Gracias!
> > Alex
> 
> Hm. I feel like if we could detect the conditions that cause this bug, we
> would be most of the way to understanding how to fixing it properly, as
> opposed to disabling the sandbox.

Unfortunately, having automatic disabling of the sandbox is not something we'd really want to put in place.
Turning it off if the content process failed to start would be open to abuse by attackers.

Even if we knew the conditions (and couldn't fix it to work with them), it would be a risky strategy in my opinion.
We'd probably more likely want to produce some sort of help page to give details of what needed changing in the environment to get it working.

> bobowen - suppose you had the computer sitting next to you at your desk. How
> would you approach solving this problem?

Given that even with the printf logging we're not getting anything, I'd probably be looking to step through the process start-up in windbg.
Which would probably mean having admin access on the machine, if only to install.

If we can't get on to an actual machine (with the ability to install tools), then a set of VM installation steps that reproduce the problem would be great.
I'll gladly set up a VM here for this, it will be useful for testing as we look to strengthen the sandbox further.
Flags: needinfo?(bobowencode)
Hi Alex, would you mind testing the two Nightly Try builds linked below please.
I created them for a different bug, but it's possible that they might help here.

Also, have you got any further with instructions on setting up a VM to reproduce this?


https://archive.mozilla.org/pub/firefox/try-builds/bobowencode@gmail.com-10b87d69fce9ddb6d683be4696eaf2705333dcc1/try-win32/firefox-53.0a1.en-US.win32.installer.exe

https://archive.mozilla.org/pub/firefox/try-builds/bobowencode@gmail.com-a0670168ea3c088771f5d311e9f7050f350dd89f/try-win32/firefox-53.0a1.en-US.win32.installer.exe
¡Hola Bob!

Life prevented me from getting to this earlier and now the links give me 404s.

I did reach out to the team that did this virtualization but they did not get back to me, sorry.

Today's Nightly still fails unless security.sandbox.content.level is set to 0.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga)
(In reply to alex_mayorga from comment #35)
> ¡Hola Bob!
> 
> Life prevented me from getting to this earlier and now the links give me
> 404s.

Hi Alex, yes we don't keep those around for too long, but one of those fixes is in Nightly anyway.
 
> I did reach out to the team that did this virtualization but they did not
> get back to me, sorry.

I'm not sure what to suggest next, without access to a machine on which we can reproduce.

Are you able to use something like sysinternals Process Monitor to log things from the Firefox processes?
I'm guessing you possibly can't as I think it needs admin access.

I'll needinfo myself to remind me to try and create a build with some more printf debugging in it, to see if we can find any clues at all.
Flags: needinfo?(bobowencode)
Mass wontfix for bugs affecting firefox 52.
Hi Alex,

Sorry I've not been able to look at this for a while.

Just checking, could you try the build that I posted in bug 1360029 on these VMs:
https://archive.mozilla.org/pub/firefox/try-builds/bobowencode@gmail.com-385e685658ee0e213cd9fa1c405c2c06cb984bab/try-win64/firefox-56.0a1.en-US.win64.installer.exe

Could you test with security.sandbox.content.level set to 0 (to make sure it works without the sandbox), 1 and 2.
Thanks.

(Also, thanks for the crashes you posted on bug 1360029 by the way, it was on those that I spotted that DLL that looks like it's related to most of those crashes.)
Flags: needinfo?(bobowencode) → needinfo?(alex_mayorga)
¡Hola Bob!

Finally got a chance to look into these again.

The only thing that made Nightly work was setting MOZ_DISABLE_CONTENT_SANDBOX to 1 as an environment variable.

¡Gracias!
Alex
Flags: needinfo?(alex_mayorga) → needinfo?(bobowencode)
Hi Alex, I haven't come up with any new ideas or seen anything else that we've fixed that looks similar, but I realised that we still haven't managed to get any logging out to try and work out where in the child process start-up is failing.

I've added some simple printf logging to a try push:
32-bit - https://queue.taskcluster.net/v1/task/IqXlFkhlTBSeLOAymAK_qg/runs/0/artifacts/public/build/install/sea/target.installer.exe
64- bit - https://queue.taskcluster.net/v1/task/YDi7ZzxHRQSenjZMCcAeJg/runs/0/artifacts/public/build/install/sea/target.installer.exe

You'll need to redirect to a file to get the logging out of the sandbox child processes, so run it from a command line something like this:

firefox.exe -no-remote -P profile_name > bug1316665.txt


I still think this may be very hard to track down without getting access to one of the VMs.
Flags: needinfo?(bobowencode) → needinfo?(alex_mayorga)
Priority: -- → P3
Whiteboard: sb? → sb+
¡Hola Bob!

Thanks on looking into this again.

Will try these builds this week.

Do you have any idea as to why ESR might have gone crash happy too now as well?

Please check the crash reports below and let me know if I'm doing anything wrong there.

¡Gracias!
Alex

ID segnalazione 	Data invio
bp-55b92084-5da0-4da5-9a06-08d0c0180221
	21/2/18	12:57
bp-d7bbb56c-fcb7-4079-bef1-c9ac80180221
	21/2/18	12:57
bp-947bf533-8935-4f71-86a5-55e840180221
	21/2/18	12:57
bp-53c38e66-4542-4a05-83c6-d23c10180221
	21/2/18	12:53
bp-ba59eef7-91ae-4fdd-a2af-44bf60180221
	21/2/18	12:53
bp-d6a410b8-f553-4ba9-9188-c3ce90180221
	21/2/18	12:52
bp-ccb88316-d35a-4048-b3ea-4e1e00180221
	21/2/18	12:52
bp-9f7b93db-0ecc-4864-8575-ffd9f0180221
	21/2/18	12:52
bp-22d4ea64-de9c-46d5-9c57-d7c240180221
	21/2/18	12:51
bp-395a01e7-3576-4880-90bc-8f7410180221
	21/2/18	12:50
bp-86198437-9cb7-46da-93a7-9397f0180221
	21/2/18	12:49
bp-1ed772d4-c5e2-40d2-9eec-ca3f10180221
	21/2/18	12:30
Flags: needinfo?(alex_mayorga) → needinfo?(bobowencode)
¡Hola Bob!

I can't download.

This is the error from that other browser from the A-Z company:

This site can’t be reached
The webpage at https://queue.taskcluster.net/v1/task/YDi7ZzxHRQSenjZMCcAeJg/runs/0/artifacts/public/build/install/sea/target.installer.exe might be temporarily down or it may have moved permanently to a new web address.
ERR_TUNNEL_CONNECTION_FAILED

Perhaps the proxy is blocking .exe files now...

¡Gracias!
Alex
(In reply to alex_mayorga from comment #43)
...
> This site can’t be reached
> The webpage at
> https://queue.taskcluster.net/v1/task/YDi7ZzxHRQSenjZMCcAeJg/runs/0/
> artifacts/public/build/install/sea/target.installer.exe might be temporarily
> down or it may have moved permanently to a new web address.
> ERR_TUNNEL_CONNECTION_FAILED
> 
> Perhaps the proxy is blocking .exe files now...

Hmm, the link still works for me. I think you'll have to speak to the people who run the proxy about this.

(In reply to alex_mayorga from comment #42)
...
> Do you have any idea as to why ESR might have gone crash happy too now as
> well?

I don't I'm afraid, looks like some sort of crash due to a hang, but it's not clear that it is related to this issue.
Flags: needinfo?(bobowencode)

¡Hola Mike, Bob!

Resolving invalid as I no longer have access to the problematic environment.

¡Gracias!
Alex

Status: NEW → RESOLVED
Closed: 8 months ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.