1708868 - Slow Upload Speed When Uploading a Large File to Google Drive When HTTP/3 Is Enabled

Status: NEW

(Core :: Networking: HTTP, defect, P2)

Description

[SL]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0

Steps to reproduce:

1. Log in to Google Drive.
2. Upload a large file (I used a 128 MiB file here) to Google Drive.

Actual results:

The upload status will be stuck at "Starting upload...", and while the upload DOES get uploaded eventually, it takes an absurdly long time for it to do so (~30 seconds to upload a 128 MiB file, which equates to ~4.47 Mbps).

However, I firmly believe my internet connection speed isn't the problem here -- it usually goes upward of 200 Mbps upload. I tried uploading the same file to Google Drive using Edge (Chromium) and there was no such bug -- the upload completed within less than 5 seconds, and the upload status is reported.

An interesting observation I noticed was that this bug was reproducible in the first 6 beta releases from the 88 cycle (88.0b1 - 88.0b6) but was not reproducible in the remaining beta releases and the two release candidates (88.0b7 - 88.0b9, 88.0rc1, 88.0rc2). HOWEVER, this bug got reintroduced in 89.0b1 and is still reproducible in the current beta (89.0b6), which hints at the possibility that this bug was caused by code within the EARLY_BETA_OR_EARLIER define.

Expected results:

The upload should complete within a reasonable amount of time, and the status should be reported.

Comment 1

[SL]

Attachment: Using Edge to upload 128 MiB to Google Drive

Comment 2

[SL]

Attachment: Using Firefox to upload 128 MiB to Google Drive

Comment 3

[SL]

I ran mozregression using the mozilla-central repository and got the following regression range: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=f305437a586028e637dbe2bd7e2c29bbe45c73fd&tochange=76a285c237e9f9c6bf7ebc0b40831d560895a3f5. Turns out the offending patch was bug 1699490, which enables HTTP/3.

As a confirmation, I switched the network.http.http3.enabled preference to false, and as expected, the issue doesn't reproduce.

However, I don't think switching off HTTP/3 is a viable solution to this problem, so please find a fix to this issue as soon as possible, since uploading large files to a commonly used service like Google Drive does seem like a common use case, so I'm needinfo'ing the patch author of the offending patch here.

---

Shown below is the full mozregression results:

app_name: firefox
build_date: 2021-03-20 02:31:12.834000
build_file: C:\Users\SL\.mozilla\mozregression\persist\76a285c237e9-shippable--autoland--target.zip
build_type: integration
build_url: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Tcwr8dBlRxiKFuWNotumJg/runs/0/artifacts/public%2Fbuild%2Ftarget.zip
changeset: 76a285c237e9f9c6bf7ebc0b40831d560895a3f5
pushlog_url: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=f305437a586028e637dbe2bd7e2c29bbe45c73fd&tochange=76a285c237e9f9c6bf7ebc0b40831d560895a3f5
repo_name: autoland
repo_url: https://hg.mozilla.org/integration/autoland
task_id: Tcwr8dBlRxiKFuWNotumJg

Comment 4

[Dragana Damjanovic [:dragana]]

I cannot reproduce the issue. I do no have as fast connection as the reporter, it also sounds that hat may not be an issue here.

Can you send me a HTTP log:
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

Comment 5

[SL]

Attachment: log.txt-main.15420 (HTTP3 enabled).7z

Comment 6

[SL]

Attachment: log.txt-main.1316 (HTTP3 disabled).7z.001

Comment 7

[SL]

Attachment: log.txt-main.1316 (HTTP3 disabled).7z.002

Comment 8

[Jeronimo Torti (Please send NI request to Brindusa Tot)]

Hi SL,
I have tried to reproduce the issue with on Windows 10, on Firefox Nightly 90.0a1 (2021-05-11) (64-bit) and Release 88.0.1 (64-bit) versions.
Could you answer the following questions in order to further investigate this issue.

1. Does this issue happen with a new profile? Here is a link on how to create a new profile: https://support.mozilla.org/en-US/kb/profile-manager-create-remove-switch-firefox-profiles

Can you test the issue while in Safe Mode. You can find helpful info here : https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode.

2. Does this issue occur in the latest nightly version of firefox? Here is a link from where you can download it: https://www.mozilla.org/en-US/firefox/channel/desktop/

3. If you are still able to reproduce this issue, can you send a HTTP log, as it was requested in Comment #4 ?

Thanks in advance, I've assigned a component in order to get the dev team involved.

'Firefox - File Handling' team: if the component is not relevant please change it to a more appropriate one.

Comment 9

[SL]

I tried to reproduce this bug again, and it seems that it is only partially fixed by bug 1709166 (I got this regression range from running mozregression to find the fix). However, while the upload progress does get shown when uploading the file to Google Drive, the upload speed didn't improve at all -- it still takes ~30 seconds just to upload a 128 MiB file.

1. Does this issue happen with a new profile? Here is a link on how to create a new profile: https://support.mozilla.org/en-US/kb/profile-manager-create-remove-switch-firefox-profiles

Yes.

Can you test the issue while in Safe Mode. You can find helpful info here : https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode.

The bug still reproduces in safe mode.

2. Does this issue occur in the latest nightly version of firefox? Here is a link from where you can download it: https://www.mozilla.org/en-US/firefox/channel/desktop/

Yes.

Comment 10

[SL]

Attachment: log.txt-main.1332 (HTTP3 enabled).7z

Comment 11

[SL]

Attachment: log.txt-main.10036 (HTTP3 disabled).7z.001

Comment 12

[SL]

Attachment: log.txt-main.10036 (HTTP3 disabled).7z.002

Comment 13

[SL]

Attachment: Uploading 128 MiB to Google Drive (HTTP3 enabled).mp4

Comment 14

[SL]

Attachment: Uploading 128 MiB to Google Drive (HTTP3 disabled).mp4

Comment 15

[Dragana Damjanovic [:dragana]]

I will not have time to work on this bug, so I am unassigning myself.

Comment 16

[zaxa321]

It reproduces on my end, and not only for google drive, this seems to be true for all http3 uploads on windows. It reproduces only on windows and not on linux (everything works as expected there). Switching 'network.http.http3.enable' to false does mitigate the issue, but only because http3 protocol is not used anymore.

Comment 17

[taylor]

Hello all, I am sharing my experience with this bug too. I am getting about 50% or less upload speeds with Firefox compared to Chromium with HTTP3 (for example Google Drive uploads).

I just did a test on Google Drive, and using the exact same file, it would take Firefox 21 minutes to upload, whereas using Edge (Chromium) it would take 7 minutes. I measured that Firefox was at max 300Mbps, while Edge was at about 600Mbps.

Here is an image showing the upload times as well as the bandwidth: https://i.ibb.co/X3Zv8xX/Firefox-vs-Edge-Speed.jpg

I have a 1 gigabit fiber connection and am running on Windows 10 (most up-to-date version as of 6-14-23)

Comment 18

[Andrew Creskey [:acreskey]]

I'm seeing similar problems on Windows, on a slower network (45Mbps upload).

Comment 19

[Andrew Creskey [:acreskey]]

We landed http upload telemetry before the fix to Bug 1842923 and it also appears that in general H3 upload performance is the slowest of the HTTP/1, HTTP/2, and HTTP/3

https://glam.telemetry.mozilla.org/firefox/probe/http_upload_bandwidth_mbps/explore?channel=release&currentPage=1&process=parent

Comment 20

[Andrew Creskey [:acreskey]]

If anyone seeing this performance discrepancy can help test out a hypothesis, it would be appreciated.

In bug 1851908, we just introduced a preference named network.http.http3.cc_algorithm, which can be accessed through about:config. This preference exposes the congestion control algorithm employed for HTTP/3.

Please follow these steps:

Make sure you see the preference `network.http.http3.cc_algorithm` (Firefox Nightly 119 and onwards)
Measure the upload throughput while on the default setting (`1` for Cubic).
Modify this preference to `0` (indicating newReno).
Restart Firefox and measure the upload throughput again.

Comment 21

[Kershaw Chang [:kershaw]]

Hi,

Please have a look at this comment.
We would greatly appreciate it if someone could assist us in collecting some logs and also check if the upload performance has improved.