Open Bug 1713716 Opened 3 years ago Updated 1 year ago

Prevent auto-resumption of downloads at startup

Categories

(Firefox :: File Handling, enhancement)

Firefox 88
enhancement

Tracking

()

People

(Reporter: from-mozilla, Unassigned)

Details

Attachments

(3 files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0

Steps to reproduce:

Start downloading any large file (10 GB or larger) from Google Takeout
browser crashes / OOM killed / system powers off / etc
Check for a ".part" file, note that a large portion of the download has been completed
Restart browser

(the remaining steps are my intent; they have no bearing on the outcome, because by this point the fault has already occurred)
Log into Google Takeout
Attempt to re-start download

Change browser.download.manager.resumeOnWakeDelay to a much larger number, and try again next time it fails.

Actual results:

Background: Firefox crashes on average once every 8-10 GiB during downloads, but the same thing happens however Firefox gets restarted. Other activity influences this, but even just leaving the computer completely untouched it rarely gets past about 12GB.

Before I even get to log into Google Takeout, Firefox has attempted to resume the download, but without the necessary credentials.

This results in Takeout sending a 68-byte file that says "domain expired", and the previous partial download is discarded.

This results in repeated attempts to download the same file, often resulting in Google refusing saying "too many download attempts".

Setting browser.download.manager.resumeOnWakeDelay to 1000000 does not seem to have any effect. I would expect that to make it wait about a quarter of an hour before resuming downloads, but they seem to resume anyway.

My current work-around is:

  1. before restarting Firefox, move the .part files so that Firefox can't remove them
  2. start Firefox, wait 10 seconds for it to auto-resume
  3. delete any 68-byte files
  4. request download again
  5. wait until it appears in the download drop-down menu, then hit the "×" to pause the download
  6. wait 5 seconds; move the previous .part file so that it replaces the much smaller, new .part file (this works even if you use a different "save as" name)
  7. in the download drop-down menu, hit the circular arrow to resume

The delay makes this more reliable; I'm not sure why.

Expected results:

This is both a bug report and a feature request:

bug report: Setting browser.download.manager.resumeOnWakeDelay to a huge number should effectively prevent downloads from automatically resuming.

feature request: please provide some UI managing partial downloads upon restart; I suggest making it part of about:sessionrestore; in any case, don't auto-resume until after the user responds to about:sessionrestore.

I don't even care about having to mess about renaming the .part files; what's important is that I don't lose them simply by restarting firefox before I realise that they're still pending.

This is important because I need to download dozens of files that take in excess of an hour each, within a six-day limit.

The six-day limit is important because a Takeout image comprises multiple tarballs or zip files, and the grouping of images, videos, emails, etc within those files is essentially random each time (it's a bin-packing problem to bring each file as close to but not exceeding the requested size limit, in my case 10 GB), so it's not simply a matter of making a new Takeout image and starting from the part number where I left off.

The "domain expired" is because the URL for each part of the Takeout image changes each time it's requested, as a security measure to prevent replay attacks.

The Bugbug bot thinks this bug should belong to the 'Firefox::File Handling' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → File Handling

I mentioned a 68-byte file; this takes the form Locked Domain Expired: Not valid after ????-??-??T??:??:??.???-0?:00 (where the last part is -07:00 or -08:00 for the US Pacific Timezone, or "Mountain View Standard Time"). These bytes are inside a file whose name ends with .zip or .tgz.

Log into Google Takeout
Attempt to re-start resume download

I should have said "resume" rather than "re-start", as the difference is quite important in this context.

Each part takes 70-80 minutes if Firefox isn't also doing anything else, and will usually complete without interruption, while two simultaneously will take about 120-130 minutes, but increases the likelihood of Firefox crashing before either can complete.

There is a limit on the number of attempts that can be made to download each part, about six though I'm not certain as I only ever encounter it when other things are already going wrong. As a result downloading more than 4 parts concurrently pushes up the number of restarts required - even using my pseudo-resume method - to the point where exceeding the per-part limit on download attempts becomes likely or inevitable.

A complete download takes of the order 2 DAYS when it goes WELL, or 3-4 if I have to manage interruptions and resume downloads.

There is a limit of six days to download all parts of the Takeout image, so there isn't much leeway for "problems".

If I exceed either limit that drastically reduces the utility of the entire download as it fails to be a complete backup.

Whilst in theory I could elect to take the download in smaller pieces, to reduce the loss if parts have to be restarted, that means I effectively had a full-time job for 3 days just hitting "download next" every few minutes. Me taking a coffee break results in lost download time.

While the "mean time between crashes" is about an hour, or about 8GB, I've seen it abort as early as 25 minutes or 3GB, and - just once - managed to download two full 10GB parts consecutively without crashing on either.

I'm confused why the User-Agent says x86_64 when this is the 32-bit version:

$ file -L /usr/bin/firefox
/usr/bin/firefox: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=cb9f59d390f9a888e55fcbc2ea2fa2ec69183094, stripped
$ dpkg -l libc6:i386
...
ii  libc6:i386                             2.23-0ubuntu11.2         i386                     GNU C Library: Shared libraries
$ uname -ioprsv
Linux 4.4.0-140-generic #166~14.04.1-Ubuntu SMP Sat Nov 17 01:52:43 UTC 2018 x86_64 x86_64 GNU/Linux
$ free
             total       used       free     shared    buffers     cached
Mem:       8165492    7909328     256164     200820     465720    4708768
-/+ buffers/cache:    2734840    5430652
Swap:      5001212     309040    4692172
$ grep model /proc/cpuinfo 
model           : 15
model name      : Intel(R) Core(TM)2 Duo CPU     T7100  @ 1.80GHz
model           : 15
model name      : Intel(R) Core(TM)2 Duo CPU     T7100  @ 1.80GHz

Oh, and I should probably mention: I have a Linux cgroup that restricts Firefox to 4GiB of memory (across all instances combined), to ensure that it can't go into a swapping frenzy, or starve Xorg or other critical processes.

And no, I can't simply upgrade the OS, because I've yet to find a newer release that supports my graphics card.

Hi Martin, I will change the type of this Bug bug to Enhancement/Request based on:

  1. bug report: Setting browser.download.manager.resumeOnWakeDelay to a huge number should effectively prevent downloads from automatically resuming.
  2. feature request: please provide some UI managing partial downloads upon restart; I suggest making it part of about:sessionrestore; in any case, don't auto-resume until after the user responds to about:sessionrestore.

Can you please also share the about:support info? Unfortunately, I don't have the needed setup to check this out. Please also share the crash report, even if it is an OOM crash, it might have some useful info that could fix this crash in the first place. You can find the crash report in about:crashes, please make sure to submit it and send over that link.
Thanks!

Status: UNCONFIRMED → NEW
Type: defect → enhancement
Ever confirmed: true
Flags: needinfo?(from-mozilla)
Attached file about:support
about:crashes shows my most recent crash as 3 May 2021 - about 5 weeks ago. However Firefox shut down and had to be restarted within the last 10 minutes. about:support says:

(Oh great, "upload as attachment" ALSO submits my half-written comment, with no way to go back and edit it.
I'm afraid github is now setting a high bar as the minimum standard for all bug trackers.)

I was going to say, I finally finished my 3-day download a few days ago, and Firefox has since auto-updated from 88.0.1 to 89.0, so I'm not certain whether the about:support will be useful, but I've attached it anyway.

If I run Firefox from a shell, I get random reports about widgets, and usually the last thing before the shutdown is something about a channel closing, rather than termination by a signal or exception.

As soon as I've finished writing this, I will restart Firefox from a terminal, and the next time it stops I will add another report here.

Flags: needinfo?(from-mozilla)

That crash report was from 3 May (5 weeks ago), which now I think about it, corresponds to (shortly before) when I upgraded the RAM from 2.5 GiB to 8 GiB, and increased the Firefox cgroup limit from 1.5GiB to 4GiB.

So those crashes were very likely related to memory pressure, but (because of the cgroup) would have been malloc/sbrk/mmap(ANON) failing, rather than the kernel's OOM-killer on the loose.

Although it's no longer "crashing" (merely quitting without me asking it to) I suspect it's still encountering the (revised) memory limit.
I will try to add some instrumentation to periodically check the "available memory" in that cgroup.

I said I would report back the next time Firefox stopped. The few lines of output were: ``` libGL error: failed to open drm device: Permission denied libGL error: failed to load driver: i965 ** (firefox:9331): WARNING **: No marshaller for signature of signal 'PropertiesChanged' ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost ``` However that's not the only time that the last line occurs in the output.
I said I would report back the next time Firefox stopped. The few lines of output were: ``` libGL error: failed to open drm device: Permission denied libGL error: failed to load driver: i965 ** (firefox:9331): WARNING **: No marshaller for signature of signal 'PropertiesChanged' ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost ``` However that's not the only time that the last line occurs in the output.

(In reply to Timea Cernea [:tbabos][inactive] from comment #6)

Can you please also share the about:support info? Unfortunately, I don't have the needed setup to check this out. Please also share the crash report, even if it is an OOM crash, it might have some useful info that could fix this crash in the first place. You can find the crash report in about:crashes, please make sure to submit it and send over that link.

See my comments above.

Until version 115 things had been quite stable, but since version 116 I'm seeing a lot more crashes and abrupt shutdowns due to memory pressure. I save stdout/stderr from Firefox into a log file, and since 2023-08-03_11:32:52+1000 the term [GFX1-]: wr_renderer_render: OutOfMemory has appeared 1200 times.

These are my submitted crash reports; I note that they are only a small fraction of incidents, as often Firefox dies (or has to be killed) without getting a chance to generate a crash report:
bp-2faff6a6-ffe1-4278-9a78-d5a2e0230622 6/22/23, 12:10
bp-a5ca0d1c-f4f3-4cb2-b847-4acbc0230617 6/17/23, 17:26
bp-3d872912-2d77-44bb-93b3-e581a0230617 6/17/23, 17:20
bp-dfe343dc-9d1f-4378-9e35-f49520230617 6/17/23, 16:18
bp-e4ca1cd0-2841-4a83-aae4-6145e0230617 6/17/23, 16:17
bp-d8a766bd-5a0d-4029-b2a5-a7e830230617 6/17/23, 16:14
bp-f669651f-3e6e-440e-bc77-a857b0230617 6/17/23, 16:14
bp-2deb8e28-d944-400f-b322-715940230606 6/6/23, 13:20
bp-ad4e85bc-7062-4716-9d83-70c950230601 6/2/23, 04:24
bp-576f160a-50a8-4951-bd68-901eb0230601 6/2/23, 04:21
bp-fea5ba55-7275-4e78-b470-277450230601 6/2/23, 04:21
bp-09f1f355-849f-4a53-a183-3b1110230309 3/9/23, 15:19
bp-cd85dd1f-c6ca-44d4-8768-6c7ec0220810 8/10/22, 10:49
bp-214e01a0-4604-4398-accd-1a6e80210928 9/28/21, 10:13
bp-335440c1-52ad-40c1-9743-ccd8a0210928 9/28/21, 10:13
bp-f221bf6f-c35b-4207-bf1a-abc650210928 9/28/21, 10:13
bp-8d897fc7-5d48-4729-842a-cb77a0210928 9/28/21, 10:13
bp-61cd04ea-62c8-4243-a5d7-01c090210928 9/28/21, 10:13
bp-0884c738-5679-40ec-913b-1b67b0210928 9/28/21, 10:13
bp-43289927-5ae5-47b9-853c-3c5680210928 9/28/21, 10:13
bp-cf48d420-6592-438f-b3b6-768fb0210928 9/28/21, 10:13
bp-9c231e85-e248-4f44-ad19-d54890210928 9/28/21, 10:13
bp-bbd5fe69-423c-4e5e-8b13-38a460210928 9/28/21, 10:13
bp-e66f8421-538f-4393-bbb0-982350210819 8/19/21, 11:15
bp-6a7bf943-8cad-4321-b6f8-7bb610210813 8/13/21, 10:32
bp-e74945c5-aa88-402f-89a2-ab80c0210607 6/8/21, 00:29
bp-1a031998-33c5-4997-9673-d4a800210505 5/5/21, 15:41
bp-8735ab2a-7c80-4036-b9cf-74d020210412 4/13/21, 09:55
bp-b7403e8d-225a-4d4f-a696-c86b50210103 1/3/21, 11:08
bp-abc69664-9542-4971-aced-594c80201231 1/1/21, 09:57
bp-f6ae73b4-64cd-4c93-95d5-c2dac0201228 12/28/20, 12:10
bp-5c401f46-ea82-4c1d-91c1-718a50201214 12/14/20, 10:53
bp-bd6e583f-eaf1-4dd8-8103-aaa620201012 10/12/20, 14:51
bp-a1998c2b-3d02-4560-8650-4f0710201012 10/12/20, 14:44
bp-4833428b-df50-4eef-b8d6-e81c00201012 10/12/20, 14:44
bp-5d57accd-aa03-4f74-84ec-57d230201012 10/12/20, 14:44
bp-c9ff636c-12b6-424f-84e6-1d4950200917 9/18/20, 07:45
bp-ea255756-692d-436c-9dd8-730090200906 9/6/20, 10:41
bp-e7b8913c-203d-4864-9ff2-a02ce0200629 6/29/20, 11:43
bp-31b5abfa-1d87-406a-87b2-c8c2d0230809 8/9/23, 14:48
bp-55409764-cb90-4546-b333-2620e0230809 8/9/23, 14:48
bp-6980546e-9313-46bd-b53b-ef61f0230809 8/9/23, 14:48
bp-b242d8f8-41e1-4823-a7d5-f49970230809 8/9/23, 14:48
bp-b7098dd5-39e7-4565-991e-1f6590230809 8/9/23, 14:48
bp-a107ca8f-c730-409c-9d65-a48470230809 8/9/23, 14:48
bp-274947d6-1fc0-4c38-8301-2fe340230809 8/9/23, 14:48
bp-cc5f5da1-d1bd-438a-a74a-96a7e0230809 8/9/23, 14:48
bp-221db55f-803f-4839-a285-149d50230809 8/9/23, 14:48
bp-89d378fc-a1f2-4b7d-b4db-76b980230809 8/9/23, 14:48
bp-84b267b1-686b-4349-afc2-6dd930230809 8/9/23, 14:48
bp-aefb1101-c213-41f4-909a-e82fb0230809 8/9/23, 14:48
bp-6c72b08b-a04e-4918-9f20-6e6820230809 8/9/23, 14:48
bp-13ef43ce-aac1-44ce-a8c2-0aa1e0230809 8/9/23, 14:48
bp-40a4c536-ba3b-40af-a275-1bba80230809 8/9/23, 14:48
bp-e2d0e8ef-e716-4cb0-a556-a13d30230809 8/9/23, 14:48
bp-040db3a9-8b05-4953-8e66-54ee80230809 8/9/23, 14:48

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: