1670407 - browser.downloads.erase doesn’t work reliably

Status: RESOLVED INCOMPLETE

(WebExtensions :: General, defect)

Description

[hans_squared]

Attachment: Test Extension

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0

Steps to reproduce:

Firefox version:
Firefox 81.0 (64-Bit)
Firefox Developer Edition 81.0b9 (64-Bit)

Your OS and version:
Manjaro Linux with XFCE
(ML uses a rolling release model; my version is up-to-date)

Attached test extension:
When you click the browser action, the test extension downloads a file (which is packaged with the extension), into the standard download directory.
When the file has finished downloading, the extension removes the file from disk and then erases it from the download history.

Actual results:

Firefox 81.0 (64-Bit):
When I load the extension with
about:debugging > This Firefox > Load Temporary add-on
everything works fine.

But when I run the extension with
web-ext run
the downloaded file remains in the download history.
But:
If I then click the browser action again, the old file is removed from the download history, and only the new file is now in the download history.
And if I click the browser action multiple times in quick succession, all files disappear from the download history.
After that, the extension works like it should: When I click once on the browser action, the file is both removed from disk and from the download history.

Firefox Developer Edition 81.0b9 (64-Bit):
The bug occurs in both cases:
about:debugging > This Firefox > Load Temporary add-on
web-ext run

Expected results:

The downloaded file should always be removed from the download history.

Comment 1

[Alex Cornestean]

Hello,

I’ve managed to reproduce the issue based on the provided STR on the latest Beta (82.0b9/20201008183927) and Release (81.0.1/20200930150533) under Windows 10 Pro 64-bit and Ubuntu 16.04 LTS. On the latest Nightly the issue could not be reproduced i.e. the downloaded file is erased from the download history as well.

On both Beta and Release, I’ve noticed the behavior described here:

“the downloaded file remains in the download history.
But:
If I then click the browser action again, the old file is removed from the download history, and only the new file is now in the download history.
And if I click the browser action multiple times in quick succession, all files disappear from the download history.
After that, the extension works like it should: When I click once on the browser action, the file is both removed from disk and from the download history.”

As this does not occur in Nightly, most likely the issue causing this behavior was fixed. I’ll search for a fix window as soon as possible. (at the moment I’m having some technical issues with the mozregression software)

Comment 2

[Alex Cornestean]

Hello,

I’ve performed a bisection to search for a fix window and narrowed it down to https://bugzilla.mozilla.org/show_bug.cgi?id=1658229 (2020-10-20T11:03:33.520000: DEBUG : Found commit message: Bug 1658229 - Exclude moz-extension:// urls from being saved in history. r=mak) with this Differential Revision: https://phabricator.services.mozilla.com/D91767 .

This is the corresponding pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=679f251515f726d69159b846875e9e676169b3ef&tochange=76f2bb2215b501a4414a090b07c5821647ddd5d1.

Will this suffice or will a regression range also be needed? Thank you!

Comment 3

[bplu4t2f]

Just stumbled across this bug.

I believe it has something to do with whether the file being downloaded is currently cached or not.

I noticed that when I download an image that is currently being displayed in a tab, and then use downloads.erase, it seems to work.

However, if I download a file from an unknown URL ("out of the blue", link copied from another machine), then it doesn't work. It just moves the download item to the bottom of the Downloads list in the Library.

This is not 100 % reliable though. I cannot pinpoint what exactly causes it.

I was able to reproduce this simply by typing browser.downloads.download / browser.downloads.erase into the Console (about:debugging -> Inspect) of any extension that has download permissions.

Comment 4

[Rob Wu [:robwu]]

This could be a race condition between Firefox finalizing the download and the extension removing data. For example, I know that an empty file can sometimes be recreated on Windows, bug 1654819.

To check whether your issue is caused by that or something else, could you introduce a small delay before initiating the removal of the download? For example, by adding

await new Promise(resolve => setTimeout(resolve, 1000));

before

				await browser.downloads.removeFile(downloadItems[0].id)
				await browser.downloads.erase({id: downloadItems[0].id})

And it could also be helpful to log event details of downloads.onChanged, to see if the issue is potentially caused by differences in reported event details.

Comment 5

[bplu4t2f]

In my experience, the problem also occurs (same probability) when there is a significant delay between downloading the file and calling erase (10 seconds, varying slightly).

Comment 6

[Rob Wu [:robwu]]

Closing bug because we are unable to reproduce it, and no further information was provided that allowed us to reproduce.

If you have (somewhat) reliable reproduction steps to trigger the bug, please add add a new comment and we can re-open it.

Comment 7

[bplu4t2f]

Not able to reproduce anymore with the extenion provided, but I am able to reproduce it, without fail, with my own extension (which downloads several, bigger files from actual internet servers).

I guess it's a timing issue, maybe a race condition.

If I ever manage to isolate it I may update this post...

Comment 8

[bplu4t2f]

Quick update on some extra details when this bug happens:

It seems that the downloads actually do get removed from the popup (that pops up when left-clicking the "downloads" button), at least that's what I'm currently observing. They do not get removed from the "Library" window.

These "zombie downloads" are NOT included in the result when calling browser.downloads.search({}), and repeated attempts to erase them don't work either.

So it feels like the bug may actually be with the Library window. At least it's obvious that it uses a different underlying data source than the browser.downloads API, so it may also be the synchronization between these two data sources.

Comment 9

[bplu4t2f]

FYI I was able to reproduce the bug without fail on 3 different machines, one of those being a VM with a fresh install of Firefox (95.0.2).

All of this on Windows 10.

Preparations to reproduce:

Make sure there is an extension installed with the "downloads" permission. OP's extension will do. It just needs a working manifest and an empty background script. It can be a temporary extension installed in about:debugging, or a "real" one, such as "Cute Save Button".

I would recommend customizing the "Downloads" button in Firefox' toolbar to DISABLE the "Hide button when empty" option. Though this doesn't seem to be necessary, I have a feeling that it might disrupt the reproduction attempt if this option is enabled.

I would recommend clearing the download history in the Library window, and restarting Firefox. This isn't necessary, but it makes it easier to track whether it was successfully reproduced.

Make sure the "Library" window is closed before proceeding.

Steps to reproduce:

(1) Go to about:debugging -> "This Firefox", find the extension with the downloads permission -> "Inspect" -> "Console". Be ready to type stuff in the console.

(2) Head over to safebooru.org to find some random images to download (NOTE: I was able to repro on other sites, but I had 100 % success rate so far on safebooru.org, so I'll recommend that site for now). https://safebooru.org/index.php?page=post&s=list

(3) Locate any image that you think you never downloaded before (this is important!). For example, this one: https://safebooru.org/index.php?page=post&s=view&id=3770069
NOTE: Content on safebooru should be safe for work.

(4) Right-click the main image on the page, and Copy Image Link. For the above example, the link will be https://safebooru.org//samples/3613/sample_cbdcc45456b1a1a59f67cc3ea71f383f201c3aeb.jpg?3770069
IMPORTANT: Do NOT open or download that link yet! Just keep it in the clipboard, don't do anything else with it.
(NOTE: Specific to safebooru.org, it doesn't seem to matter whether it's the full-size, resized, or thumbnail version of the image).

(5) Go back to the Console for the extension, and type in:
await browser.downloads.download({ url: "<PASTE IMAGE URL HERE>" })
It will return the download ID. If you've just restarted Firefox, that should be 1 (doesn't matter though).
Stay in the console for now!

(6) Observe that the "Downloads" button becomes blue, signaling that the download is complete. Do NOT interact with the "Downloads" button.

(7) Type this in the console:
await browser.downloads.erase({ id: <TYPE ID THAT WAS RETURNED HERE> })
It will return an array with 1 element, which is the erased ID.
NOTE: This doesn't need to be particularly fast. If you start typing right away, you'll probably press enter a couple seconds after the download finished. In fact, as far as I can tell, you can wait indefinitely before you type the erase command, and it'll still repro.

Observe that the "Downloads" button remains blue.
When clicking the "Downloads" button, it will say that there are no downloads for this session.
However, when opening the "Library" window by clicking "Show all downloads", the downloaded file will still be there.

As mentioned, I was able to reproduce on other sites like imgur.com, and I was also able to reproduce with things other than images, but on safebooru.org in particular, I was always able to reproduce the bug using the above sequence when I tried it.