1670149 - Extension update fails because XPIInstall.jsm does not rename xpi file when moving to trash directory

Status: RESOLVED INCOMPLETE

(Toolkit :: Add-ons Manager, defect)

Description

[mcintosh]

Attachment: Noscript update problems details.txt

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0

Steps to reproduce:

Wait for NoScript to have an update.

Actual results:

NoScript silently died, the icon was no longer showing on the browser. But about:addons continues to show it as active (blue slider with the dot on the right). Browse console logs messages about failed to remove file, failed to remove trash directory, failed to move file, failed to remove file, failed to remove trash directory. If I simply deactivate/reactivate NoScript in about:addons, everything is fine again (though still on the old version). Until the next Firefox addons autoupdate time, when NoScript dies again.

Expected results:

moveOldAddon() in XPIInstall.jsm should have either renamed the existing xpi file it was moving to the trash directory just out of principle, or it should have detected that the file it was about to move already exists in the trash directory and renamed the existing trash directory file to something else. I should never have needed to notice all this mess.

Comment 1

[Alex Cornestean]

Hello,

I’ve attempted to reproduce the issue based on the provided STR and additional info on the latest Nightly (83.0a1/20201011093320), Beta (82.0b9/20201008183927) and Release (81.0.1/20200930150533) under Windows 10 Pro 64-bit and Ubuntu 16.04 LTS, however without success i.e NoScript will properly update and will not break.

However, please note that I’ve tested this on a new profiles with no prior installed extensions. The update process (either auto-update or manual update) will update to the latest extension version and not to an intermediary version and as such there won’t be any residual .xpis which the browser is unable to delete.

I will continue checking this issue on the mentioned profiles which already has had NoScript 11.0.43 installed and updated once to the current latest version 11.1.1, as new updates for the extension are available.

Comment 2

[mcintosh]

The key is the locked file per Sysinternals Handle (or similar program). Because it's locked, it cannot be deleted. It stays in the trash directory blocking the next update. So you can always update the first time after a restart, because there's no persistently locked file in the trash directory. My current FF instance has been running since Sept 14.

I see 11.1.3 is now out, and as soon as I do check for updates, bam, away goes the NoScript icon on the windows. I assume it was added after 9pm last night, because NoScript did not auto die last night. It will auto die tonight if I don't rename the {73a6fe31-595d-460b-a920-fcc0f8843232}.xpi file in the trash directory.

Here's the handle output currently ... it's long, I put this in a new attachment. What I noticed, in 2>>, is that there were no open file handles to the currently installed .xpi file with NoScript disabled. After reactivating NoScript, I renamed the blocking file in trash to _3 and ran the update. Ran fine, as expected. Unexpectedly though, there was no new locked ....xpi file in the trash directory. Just the old _1, _2, _3 files.

So did NoScript fix something in 11.1.1? Is there any way to know why FF had open file handles to the xpi files, even when the extension was dead? Or did I clear something by manually disabling the addon (something I've not done previously in this sequence) before trying the update? I don't know. It'll take two more updates of NoScript to find out, one to move a locked xpi into the trash dir (assuming the locked xpi problem still exists) and a second to clash into that locked xpi in the trash dir.

Comment 3

[mcintosh]

Attachment: Noscript xpi open handle output.txt

Comment 4

[Alex Cornestean]

Hello,

I’ve re-tested the issue on Nightly (83.0a1/20201011093320), Beta (82.0b9/20201008183927) and Release (81.0.1/20200930150533) under Windows 10 Pro 64-bit and Ubuntu 16.04 LTS, as you mentioned a new update for NoScript was available.

The auto-update proceeded normally, without breaking the add-on. NoScript was updated from 1.11.1 to 11.1.3 with success, the extension working without problems. Please note that yesterday the extension was updated from 11.0.43 to 11.1.1.

Comment 5

[mcintosh]

If you have any insight in to why the Firefox extension thread would have stray file handles, I figure that's about the only analytical progress that will likely be made if you're not obtaining unreleased file handles in your testing. Without a left over xpi file in the trash dir after an update, there will not be any problem encountered. As you can see in my handle output, Firefox refuses to let go of multiple trash dir xpi files.

There's still the issue of the problem occurring in the first place, letting a file name conflict in a trash directory not only block the update, but break the running state of the extension. But improving that situation handling is more an enhancement than a defect, I would agree.

Comment 6

[Tomislav Jovanovic :zombie]

Can you please try to reproduce starting from a fresh profile, otherwise it's hard to debug the problem?

Comment 7

[mcintosh]

I was trying to get more specifics of the open file handles from the live problem exhibit, but I ended up trashing the running process (FF extensions process), so I had to restart FF. This was last week. I decided while I was bouncing to update FF to 81.0.2. But I was not expecting to encounter the problem again, as the upgrade from 11.1.1 to 11.1.3 happened with no stray xpi file in a trash dir.

It would seem that whatever Noscript changed sometime around 11.0.35 (early August) that exposed FF's file handle mishandling had been corrected. Or not. So the autoupdate to 11.1.4 happened last night, and now there's an 11.1.3 xpi in the trash directory.

firefox.exe pid: 10952 type: File 6F4: C:\Users\<user>\AppData\Roaming\Mozilla\Firefox\Profiles\<profile>\extensions\trash{73a6fe31-595d-460b-a920-fcc0f8843232}.xpi

Why did 11.1.1 to 11.1.3 happen cleanly? Remember I mentioned in my post that I did something I had not done before, I disabled the addon and then reenabled it and then updated it. So there's something there in that regard, but the bottom line is something must be understood about why FF has that file handle out there.

I was set to wait for 11.1.5 or whatever is next to see it conflict, but trying the debugger again, it again hung up the extensions pid. I had to disable Noscript to make pages load. But then it trashed the pid and so the trash dir file handle was lost. I was able to recover the extensions pid by reenabling Noscript. So an endless mess.

As for spending hours attempting to recreate this with a new profile, without my bookmarks, without my passwords, without my Noscript config, I'm not sure what the value of my time towards that is. Maybe if there was a manner of running such that could allow me to query the file handles of the extensions pid to compare the list the process thinks it has vs what handle says it has. We don't know if the pid even knows it has this open file handle.

But I do not have the old xpis anymore to recreate the situation. The FF reboot last week deleted all that. I see there is a list of old Noscript versions on the https://addons.mozilla.org/en-US/firefox/addon/noscript/versions/ but all I see is a remove button (for what purpose?). Are prior versions available, or would I need to obtain that from Noscript more directly?

Comment 8

[Rob Wu [:robwu]]

(In reply to mcintosh from comment #7)

But I do not have the old xpis anymore to recreate the situation. The FF reboot last week deleted all that. I see there is a list of old Noscript versions on the https://addons.mozilla.org/en-US/firefox/addon/noscript/versions/ but all I see is a remove button (for what purpose?). Are prior versions available, or would I need to obtain that from Noscript more directly?

It's not obvious at all (reported https://github.com/mozilla/addons-frontend/issues/9811), but you can download or install an older version by right-clicking on the button and choosing "Save link as" (to download) or "Open Link in New Tab" (to install).

Comment 9

[Tomislav Jovanovic :zombie]

See above message for older versions, we can't proceed without STR.

Comment 10

[Tomislav Jovanovic :zombie]

Can't proceed without STR, feel free to reopen if you manage to reproduce reliably.