1496078 - Launcher process: Nightly won't start at all with McAfee

Status: RESOLVED WORKSFORME

(External Software Affecting Firefox :: Other, defect, P3)

Description

[Itiel]

Using Windows 10 x64 on my work laptop.
Today I fired up Nightly after about a month of not using it, and it auto-updated itself. And that's when it started not to boot. No window, no message, no error, no nothing. firefox.exe starts up in the task manager and quits after 1-2 seconds.
Running Firefox 62 is all good. No relevant crashes in about:crashes there.

Probably due to my company's firewall, I can't use Mozregression to find the regression range (message: ConnectionError: HTTPSConnectionPool(host='hg.mozilla.org', port=443): Max retries exceeded with url: /mozilla-central/json-tags (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x05DAE210>: Failed to establish a new connection: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))), so I tried bisecting it manually.
And I got it:
Last good:
https://download-installer.cdn.mozilla.net/pub/firefox/nightly/2018/09/2018-09-26-14-20-55-mozilla-central-l10n/
First bad:
https://download-installer.cdn.mozilla.net/pub/firefox/nightly/2018/09/2018-09-26-22-01-46-mozilla-central-l10n/
I can consistently run the former build, and consistently reproduce the issue with the latter.
Not sure where I can get the in-between (called inbound?) builds to install and test.

All tests were made on clean profiles.
I *think* the issue originates from the McAfee, though I'm not sure how to test this hypothesis as there's no option to disable its defenses.
There's nothing special in McAfee's logs to indicate that Nightly was blocked.
McAfee agent version: 5.0.6.220
McAfee DLP Endpoint version: 11.0.400.152
McAfee Management of Native Encryption version: 4.1.1.4
McAfee Endpoint Security version: 10.5.3

Comment 1

[Matthias Versen [:Matti]]

We need the source revision of each good/bad version. 
In case you ever have to do that again open about:buildconfig and under Source->Build from: you can see the revision

I think I got the right revisions from the Info files on the "ftp".

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=48cc597db296ea5234a96321e2ce0c52094cc254&tochange=7ac88abc3c57832e7e271dede934f8935c7f83a0

There is the fix for Bug 1488554 in that range and it's very likely the cause of this together with McAfee

Comment 2

[Sylvestre Ledru [:Sylvestre]]

[Tracking Requested - why for this release]:
Potential severe issue

Comment 3

[Sylvestre Ledru [:Sylvestre]]

By the way, Itiel, bravo for this great bug report!

Comment 4

[Julien Cristau [:jcristau]]

Aaron fyi.

Not tracking for 64 because this won't ride the trains beyond nightly quite yet.

Comment 6

[Drew Willcoxon :adw]

Sorry, didn't mean to actually move this or clear the ni on Aaron.  The ni comment was:

(In reply to Julien Cristau [:jcristau] from comment #4)
> Aaron fyi.
> 
> Not tracking for 64 because this won't ride the trains beyond nightly quite
> yet.

Comment 7

[Jim Mathies [:jimm]]

Hey Adam, can you please reach out to our McAfee contacts? We expect 3rd parties to address any issues with this new security feature.

Comment 8

[Adam Stevenson [:adamopenweb]]

Reaching out to McAfee by our partner mailing list.

Comment 9

[Marco Castelluccio [:marco]]

A while ago, they said they switched to a WebExtension and stopped injecting. Sounds like it isn't fully true?

Comment 10

[Paul Raper]

A while back I reported that Nightly does not start under the latest release. This is still the case. I am currently using a version that was recommended to me: Firefox64a.1 en-GB 2018-09-26. ANy attempt to update to a current version will result in the programme not starting.

Comment 11

[Paul Raper]

PLease note, the above issue does not affect the Developer version.

Comment 12

[Itiel]

Attachment: New error

Well, that's new.
Not sure when it started though.

Comment 13

[Liz Henry (:lizzard) (relman/hg->git project)]

So far no response on the mozilla-mcafee-discuss list. I will try a needinfo and other channels to reach out.

Comment 14

[Paul Raper]

Liz, That would be great. The Nightly issue has been going on for far too long. I am still only able to use a release that was issued back in September, ALL current releases do not work.

Comment 15

[Liz Henry (:lizzard) (relman/hg->git project)]

Jim, is there anything we can do from our end to stop this from happening? 

Andrei, can your team help to determine if this is happening on 64 and 65 as well and on which platforms? Thanks!

Comment 16

[(No longer employed by Mozilla) Aaron Klotz]

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #15)
> Jim, is there anything we can do from our end to stop this from happening? 

Two things:

(1) I need to land bug 1511078. That probably won't happen until mhowell returns next week from PTO.

(2) I need to spend some quality time with a debugger and a VM running McAfee to find out why the launcher process is broken in that environment.

 
> Andrei, can your team help to determine if this is happening on 64 and 65 as
> well and on which platforms? Thanks!

The launcher process is disabled past Nightly at the moment, so it is not an issue there.

Comment 17

[Liz Henry (:lizzard) (relman/hg->git project)]

Aaron, any luck? Looks like bug 1511078 did land.

Comment 18

[(No longer employed by Mozilla) Aaron Klotz]

I don't currently have a VM set up to test with McAfee yet, but maybe if Paul updates his copy of Nightly, he can let me know.

Paul, you might see an error message the first time that you run Firefox, but if you close that down and then subsequently run it again, you should be able to start the browser.

(Note that this resets itself every time Nightly updates. I'm working on some other code to take care of that, but in this interim state you should be able to run it at least, despite the temporary annoyance).

Comment 19

[Itiel]

(In reply to Aaron Klotz [:aklotz] from comment #18)

I don't currently have a VM set up to test with McAfee yet, but maybe if Paul updates his copy of Nightly, he can let me know.

Paul, you might see an error message the first time that you run Firefox, but if you close that down and then subsequently run it again, you should be able to start the browser.

(Note that this resets itself every time Nightly updates. I'm working on some other code to take care of that, but in this interim state you should be able to run it at least, despite the temporary annoyance).

I'm able to use now the newest build of Nightly, on the affected laptop.
Any debug info you need to move this forward?

Comment 20

[Rares Doghi, Desktop QA]

Hi, According to Comment 19 and the fact that I cannot reproduce this issue on the latest Nightly build 67.0a1 (2019-02-05), I'm inclined to think this issue is Fixed, can someone update the status for it so we can mark it accordingly?

Please note that I was unable to reproduce this issue in the past but now the reporter has confirmed that the issue is fixed for him as well.

Comment 21

[Itiel]

(In reply to Rares Doghi from comment #20)

Hi, According to Comment 19 and the fact that I cannot reproduce this issue on the latest Nightly build 67.0a1 (2019-02-05), I'm inclined to think this issue is Fixed, can someone update the status for it so we can mark it accordingly?

Please note that I was unable to reproduce this issue in the past but now the reporter has confirmed that the issue is fixed for him as well.

See comment 16- bug 1511078 is about disabling the launcher process if the browser can't start etc.
Also the error message in attachment 9029187 [details] is still seen right after updating Nightly (see comment 18).

Comment 22

[(No longer employed by Mozilla) Aaron Klotz]

(In reply to Rares Doghi from comment #20)

Hi, According to Comment 19 and the fact that I cannot reproduce this issue on the latest Nightly build 67.0a1 (2019-02-05), I'm inclined to think this issue is Fixed, can someone update the status for it so we can mark it accordingly?

Please note that I was unable to reproduce this issue in the past but now the reporter has confirmed that the issue is fixed for him as well.

I would like to leave this open, because essentially the way this is "fixed" in Nightly is for the launcher process to disable itself. If you check your about:support, you will probably find the Launcher Process state set to "Disabled due to failure". In the long term, we would like to fix that, but for the purposes of release on 67, this is exactly what we want to see.

(In reply to Itiel from comment #21)

(In reply to Rares Doghi from comment #20)

Hi, According to Comment 19 and the fact that I cannot reproduce this issue on the latest Nightly build 67.0a1 (2019-02-05), I'm inclined to think this issue is Fixed, can someone update the status for it so we can mark it accordingly?

Please note that I was unable to reproduce this issue in the past but now the reporter has confirmed that the issue is fixed for him as well.

See comment 16- bug 1511078 is about disabling the launcher process if the browser can't start etc.
Also the error message in attachment 9029187 [details] is still seen right after updating Nightly (see comment 18).

Yes, you should see the error message right after updating, but subsequent runs of Nightly should work until the next update.

Comment 23

[Rares Doghi, Desktop QA]

Hi Aaron, now with the latest Firefox Versions this issue no longer occurs at all, the Launcher process shows enabled on Nightly instead of Disabled due Failure.

Comment 24

[Itiel]

(In reply to Rares Doghi from comment #23)

Hi Aaron, now with the latest Firefox Versions this issue no longer occurs at all, the Launcher process shows enabled on Nightly instead of Disabled due Failure.

Same here.

Comment 25

[(No longer employed by Mozilla) Aaron Klotz]

I also see Nightly running with full launcher process support. I'm going to close this one out.

Comment 26

[Firefox Bug Husbandry Bot]

Please specify a root cause for this bug. See :tmaity for more information.