1570396 - Windows prompting for UAC confirmation each update

Status: RESOLVED FIXED

(Toolkit :: Application Update, defect, P1)

Description

[[:philipp]]

i'm using firefox 70.0a1 64bit on windows 10 1903 and since about five days ago i always get prompted by windows to allow the updater to proceed for each automatic update that nightly is receiving.
each time it's happening the windows eventlog is logging error id 7023 (roughly translated): The "Mozilla Maintenance Service" service terminated with the following error: Illegal function.

in parallel i see multiple temporary folders created for updates in C:\ProgramData, but i don't know if this may be related.

so far i have checked that app.update.service.enabled is set to true, tried to reinstall Firefox Nightly a couple of times (also the Maintenance Service separately through the system control panel) and manually cleared the temporary files in ProgramData without success so far.
also, switching back to older builds where i haven't noticed the issue originally didn't help but i'm not aware of having changed any system-wide prefs that might affect this on my machine before this issue first started popping up.

Comment 1

[[:philipp]]

Attachment: uac prompt.png

Comment 2

[[:philipp]]

Attachment: eventlog.png

Comment 3

[[:philipp]]

Attachment: program data.png

Comment 4

[[:philipp]]

this is a capture with the ms process monitor tool of activity from the mozilla maintenance service during the updating process: https://send.firefox.com/download/97c194d8c19dbaf0/#San3et3UarrFQhZSIPu81A

Comment 5

[[:philipp]]

Attachment: maintenanceservice.log

Comment 7

[Robin Steuber (she/her) [:bytesized]]

Could you help us get a bit more information here?

Before you start, please close Firefox so it doesn't interfere with this.
Could you open the command prompt and run cacls C:\ProgramData\Mozilla?
Then could you manually invoke the maintenance service to fix the permissions of the directory: sc start MozillaMaintenance aaa fix-update-directory-perms .
Give that a moment to finish (it runs as a service, so the command returning just means that the process has started). Just a few seconds should be sufficient.
Then, please check to see if C:\ProgramData\Mozilla exists (ignore all of the ones ending with .bak and long strings of letters and numbers).
Assuming that the directory does exist now, please run cacls C:\ProgramData\Mozilla again.

Comment 8

[[:philipp]]

i've also started seeing the same symptoms on a second device after recent updates.

i performed the steps you've described but letting the service fix the permissions of the directory didn't appear to make a difference (the C:\ProgramData\Mozilla folder was still existing afterwards).

here's the output from the command line window:

C:\Users\Philipp>cacls C:\ProgramData\Mozilla
C:\ProgramData\Mozilla VORDEFINIERT\Benutzer:(OI)(CI)F
VORDEFINIERT\Administratoren:(OI)(CI)F
NT-AUTORITÄT\SYSTEM:(OI)(CI)F

C:\Users\Philipp>sc start MozillaMaintenance aaa fix-update-directory-perms

SERVICE_NAME: MozillaMaintenance
TYPE : 10 WIN32_OWN_PROCESS
STATE : 2 START_PENDING
(NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
WIN32_EXIT_CODE : 0 (0x0)
SERVICE_EXIT_CODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x7d0
PID : 11312
FLAGS :

C:\Users\Philipp>cacls C:\ProgramData\Mozilla
C:\ProgramData\Mozilla VORDEFINIERT\Benutzer:(OI)(CI)F
VORDEFINIERT\Administratoren:(OI)(CI)F
NT-AUTORITÄT\SYSTEM:(OI)(CI)F

Comment 9

[TMart]

I'm having the same issue since a couple of days. Like Philipp my OS language is German. I'm seeing it on two company machines that are managed with a Windows domain in beta 69 and Nightly 70. Not on my home machine as of now.

Comment 11

[[:philipp]]

interestingly the two duplicate bugs are from users of german language versions of windows 10 too.

Comment 12

[TMart]

I ran the steps from comment #7 and now Nightly won't download in-app updates anymore, I always have to manually get the full installer.

Comment 13

[紫]

(In reply to TMart from comment #12)

I ran the steps from comment #7 and now Nightly won't download in-app updates anymore, I always have to manually get the full installer.

Try the following:

1. Run Firefox as an administrator
2. Check for updates

With each update two new folders will be created
c:\ProgramData\Mozilla.bakX
c:\ProgramData\Mozillaxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

The Mozilla and Mozilla.bakX is empty after the update is completed and the Mozillaxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx contains the updates folder.

I move the updates folder from Mozillaxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx to Mozilla and delete the other folders.

This is my workaround.

Comment 14

[Robin Steuber (she/her) [:bytesized]]

I'm working on trying to replicate this problem.

Philipp- Are you using Antivirus software? If so, could you describe which one and what sort of configuration you are using so that I can try to replicate it?

Thanks!

Comment 15

[Robin Steuber (she/her) [:bytesized]]

Also, are you using the German version of Firefox Nightly?

Comment 16

[紫]

I use the English (US) Version of Firefox Nightly

and Windows Security (Windows Defender Security Center)

Systeminformation
Antimalware-Clientversion: 4.18.1907.4
Modulversion: 1.1.16200.1
Antiviren-Version: 1.299.1287.0
Antispyware-Version: 1.299.1287.0

Windows Defender real-time scanning for adware, PUA or PUP enabled

https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-defender-antivirus/detect-block-potentially-unwanted-apps-windows-defender-antivirus

Windows 10 Version 1903 (Build 18362.267)

Installed Language Packs

• German
• Japanese

Comment 17

[[:philipp]]

i'm using a de build of firefox nightly and no security software except the preinstalled windows defender on win10 - no exceptional system settings/configuration that i can think of either.

Comment 18

[maggus.staab]

I'm using a DE build of firefox nightly and G DATA Security in Version 14.2.1.6 (I reported the issue in a duplicated issue)

Comment 19

[TMart]

I did a completely new installation of Nightly on a completely new laptop today. Windows 10 1903, US-English language (installed from US-English ISO), US-English Nightly installed with the full package exe. Again on a company laptop that was joined to our Windows domain as a non-admin user.

The first Nightly update today again required admin permissions. Something is seriously broken here.

Comment 20

[TMart]

Forgot to mention: Antivirus is Windows Defender.

Comment 21

[Robin Steuber (she/her) [:bytesized]]

Attachment: maintenanceservice.exe

Philipp- I have made a modified maintenance service that produces more logging that should be able to shed some light on what's going on.

Please download this executable and place it in "C:\Program Files (x86)\Mozilla Maintenance Service" (first backing up the existing maintenanceservice.exe). After the next update, this should leave logs in "C:\ProgramData\MozLog". Could you attach those logs to this bug?

Don't forget to restore the original maintenanceservice.exe afterwards.
Thanks!

Comment 22

[[:philipp]]

Attachment: firefox_pid_13244.log

here you go.

Comment 23

[TMart]

I copied the maintenanceservice.exe (as admin) as suggested in https://bugzilla.mozilla.org/show_bug.cgi?id=1570396#c21 and now it seems to be working again, no UAC prompt.

Comment 24

[TMart]

Attachment: maintenanceservice-logs.zip

Comment 25

[TMart]

Attachment: firefox-pid-logs.zip

Comment 26

[紫]

What I do not understand is that the problem occurs even if the update is triggered / executed directly in Firefox with administrator privileges.

Comment 27

[紫]

I just installed the update to 68.0.2 and now have the same problem with the release version. The User Account Control has requested the confirmation that the updater is allowed to make changes to the system.

Comment 28

[Robert Strong (they/them - no direct email)]

(In reply to Geisler from comment #27)

I just installed the update to 68.0.2 and now have the same problem with the release version. The User Account Control has requested the confirmation that the updater is allowed to make changes to the system.

Open Tools > Options > type Updates in the Find box and check if "Use a background service to install updates" is checked. If it isn't check it and try again

Comment 30

[Robert Strong (they/them - no direct email)]

(In reply to Geisler from comment #27)

I just installed the update to 68.0.2 and now have the same problem with the release version. The User Account Control has requested the confirmation that the updater is allowed to make changes to the system.

Note: If this machine also has nightly installed it will also exhibit this bug

Comment 31

[Robin Steuber (she/her) [:bytesized]]

Attachment: maintenanceservice.exe

Philipp- I have a possible fix that I would like you to try.

First, make sure that the maintenance service is still enabled. It may disable itself after if failing too many times:

Open Tools > Options > type Updates in the Find box and check if "Use a background service to install updates" is checked. If it isn't check it.

Next, use this replacement maintenance service binary the same way you did last time:

Download this executable and place it in "C:\Program Files (x86)\Mozilla Maintenance Service" (first backing up the existing maintenanceservice.exe).

With this maintenance service binary, I think you should be able to update without seeing the UAC prompt. Let me know whether this works for you or not.

Don't forget to restore the original maintenanceservice.exe afterwards.

Thanks!

Comment 32

[Robert Strong (they/them - no direct email)]

If anyone else experiencing this bug could try out comment #31 it would be appreciated. Thanks!

Comment 33

[[e-mail]]

Here is one other with the problem. Like someone before Win 10 with 1903 update and GDATA as anti virus.

If tried the new file from commend #31
After a restart of Nightly and new update, there were NO new UAC notification.

Comment 34

[[:philipp]]

you're right - the checkbox to use the background service for updates became unchecked in my case.
after enabling it & replacing maintenanceservice.exe with the one provided in comment #31, the update went through smoothly without a UAC prompt. thanks for the fix!

Comment 35

[Robin Steuber (she/her) [:bytesized]]

It looks like we have identified the problem. The code that was causing it will be backed out in Bug 1486637. However, this bug has also brought some issues to light other than the UAC prompt: The failures during permission setting. It appears that the code doing this permission setting is sometimes failing in ways that I didn't initially expect. This is what is causing all those extra Mozilla directories in C:\ProgramData.

I will be posting a patch shortly that should address those failures, but will not address the UAC problem without the Bug 1486637 backout also being merged.

Comment 36

[Robin Steuber (she/her) [:bytesized]]

Attachment: Bug 1570396 - Fix misc. issues with permission setting r=mhowell

This patch fixes a number of miscellaneous issues with permission setting.
When we move a directory, recreate it, and move the contents back, those contents may be in use (possibly by antivirus). To address this, we now fall back to copying the data back and removing the originals.

The maximum number of backups to make when moving conflicting files is now 3 instead of 9. 9 seemed a bit excessive.

There is now an additional level of SetPermissionsOf, FilesAndDirsWithBadPerms. This new value causes permissions not to be fixed if we are unable to read them. This should prevent unnecessary permission fixes while still allowing us to aggressively set them when necessary.

Comment 37

[Robert Strong (they/them - no direct email)]

As discussed on irc, does this also cleanup the directories it creates on failure and is the permission code still failing with the same error?

Comment 38

[TMart]

It didn't work for me. But as it worked for everyone else and I guess I'm the only one in a corporate environment it might have a different cause in my case.

Comment 39

[Robin Steuber (she/her) [:bytesized]]

Tmart- So far, you have posted a few logs, but only of things working properly. Could you please post the maintenance service log that you see when update fails?

It might also be useful to see the Firefox update logs. You can get these by setting app.update.log to true in the "about:config" page. This step must be done prior to updating so that the logs are created. Then before restarting the browser to install (or after failing to download), open the browser console: Hamburger Menu -> Web Developer -> Browser Console. You should see entries starting with "*** AUS:SVC" describing the update process that took place.

Comment 40

[Robin Steuber (she/her) [:bytesized]]

Robert- Hmm. My latest testing indicates that the cleanup code I wrote isn't entirely working as it should. Let me look into that.

Comment 41

[Robin Steuber (she/her) [:bytesized]]

TMart- You may also need to re-enable the maintenance service if it has disabled itself:

Open Tools > Options > type Updates in the Find box and check if "Use a background service to install updates" is checked. If it isn't check it.

Comment 42

[Robin Steuber (she/her) [:bytesized]]

Robert- It appears that the cleanup code was working properly and my testing methods were causing the problem I saw.

Comment 43

[Robert Strong (they/them - no direct email)]

Did you test with a file in use? Also, what is the answer to the second question below?

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #37)

As discussed on irc, does this also cleanup the directories it creates on failure and is the permission code still failing with the same error?

Comment 44

[Daniel Veditz [:dveditz]]

The send link in comment 4 has expired anyway so I'm unhiding it. That wasn't the best use of either private comments or send links--use out of band DMs for ephemera. I have no idea what happened to comment 6 -- nothing in history about it.

Comment 45

[Robin Steuber (she/her) [:bytesized]]

Discussed on IRC

Comment 47

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2b97cab939ac
Fix misc. issues with permission setting r=mhowell

Comment 48

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/2b97cab939ac

Comment 49

[Robert Strong (they/them - no direct email)]

Note: this bug won't be fixed entirely until after bug 1486637 is backed out. It should be merged to nightly tomorrow.

Comment 50

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/d1e960d52ee7

Comment 51

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr68/rev/35a6d667978c

Comment 52

[[:philipp]]

i am still getting the UAC prompts with each nightly update for what it's worth.

Comment 53

[Robin Steuber (she/her) [:bytesized]]

Could you double check that the service is enabled?

Open Tools > Options > type Updates in the Find box and check if "Use a background service to install updates" is checked. If it isn't check it.

Could you also give us the Nightly build you are on? It can be found in about:support on the third row.

Comment 54

[Robin Steuber (she/her) [:bytesized]]

Assuming that you are indeed on a recent build with the service enabled, could I get a recent maintenance service log?

Comment 55

[[:philipp]]

ah, you are right - the checkbox was unchecked. i'm on build 20190823094538 currently and will report back after receiving the next update with "Use a background service to install updates" enabled.

Comment 56

[Marco Zehe (:MarcoZ)]

(In reply to Kirk Steuber (he/him) [:bytesized] from comment #53)

Could you double check that the service is enabled?

Open Tools > Options > type Updates in the Find box and check if "Use a background service to install updates" is checked. If it isn't check it.

Could you also give us the Nightly build you are on? It can be found in about:support on the third row.

I am also getting the UAC prompt with nightly update still, and am on Build https://bugzilla.mozilla.org/show_bug.cgi?id=1570396. The said checkbox is checked. However, the last log that I have was made on August 16. So over a week old.

Comment 57

[[:philipp]]

with the "Use a background service to install updates" preference enabled, the latest update got applied without a UAC prompt now in my case. thank you

Comment 58

[Robin Steuber (she/her) [:bytesized]]

MarcoZ- It sounds to me like the maintenance service is not running for you. That would be a different type of failure than the one being described in this bug. Could you file a new bug for this?

It would really help if you could include the update console logs for us. This is how to obtain them:

1. Go to about:config and set app.update.log to true.
2. Wait for another update to be ready to install (i.e. wait until you see the green "restart to update arrow")
3. Go to Hamburger Menu -> Web Developer -> Browser Console
4. You can filter out non-update messages by using the filter: AUS:SVC

Comment 59

[maggus.staab]

just to get this right:

• in the past we dont had a UAC window. updates worked fine - without the need for the background service
• with the current nightly we get the UAC window
• we need to manually activate the background-service which will make sure we no longer see the UAC window (I need to test this step)

why is this solution acceptable? will the background service be activate for everyone in the near future?

Comment 60

[Robert Strong (they/them - no direct email)]

No.

The background service is used when it is installed so the UAC prompt is not shown when the install location isn't writable by the user.

You don't need to manually activate the background service.

If your install location can be written to with your user account without elevation and you are seeing the UAC prompt then something is going wrong. Since this bug is fixed for some other people as well as we've reproduced it and verified that the patch fixed it please file a new bug. Thanks!

Comment 61

[Robin Steuber (she/her) [:bytesized]]

To be clear, the reason that we have been telling people to manually activate the maintenance service is because Firefox stops attempting to use the service if it fails consistently. Since this bug was causing it to fail consistently, some people may have had the maintenance service become disabled. In this case, yes, the maintenance service may need to be manually re-enabled.

But as Robert said above, the default is to use the maintenance service, if it is available and needed. Typically, you should not need to activate it manually.

Comment 62

[Robert Strong (they/them - no direct email)]

Thanks Kirk

Comment 63

[Marco Zehe (:MarcoZ)]

(In reply to Kirk Steuber (he/him) [:bytesized] from comment #58)

MarcoZ- It sounds to me like the maintenance service is not running for you. That would be a different type of failure than the one being described in this bug. Could you file a new bug for this?

Done, bug 1576815.

It would really help if you could include the update console logs for us. This is how to obtain them:

1. Go to about:config and set app.update.log to true.
2. Wait for another update to be ready to install (i.e. wait until you see the green "restart to update arrow")
3. Go to Hamburger Menu -> Web Developer -> Browser Console
4. You can filter out non-update messages by using the filter: AUS:SVC

Done, although I didn't see anything suspicious in there. It claims everything is OK, then brings up the UAC prompt nevertheless. But perhaps you can read something out of this that I am overlooking.

Comment 64

[maggus.staab]

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #60)

No.

The background service is used when it is installed so the UAC prompt is not shown when the install location isn't writable by the user.

You don't need to manually activate the background service.

If your install location can be written to with your user account without elevation and you are seeing the UAC prompt then something is going wrong. Since this bug is fixed for some other people as well as we've reproduced it and verified that the patch fixed it please file a new bug. Thanks!

thx for the reply.

I already reported my problem in bug 1571360 which was marked as a duplicate of this bug ... maybe its not a duplicate but a different problem with a similar/same UAC coming up

Comment 65

[Robin Steuber (she/her) [:bytesized]]

Clearing this needinfo. If you are still having the problem, please file a new bug so we can address it.