1689752 - DownThemAll! downloader fails downloads of password-protected link with "Server error" since v85

Status: RESOLVED DUPLICATE of bug 1691907

(WebExtensions :: General, defect, P1)

Description

[Andreas Schiffler]

Attachment: DownThemAllError.PNG

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0

Steps to reproduce:

Have DownThemAll! extension (ver 4.2.6) installed. I frequently download sets of video files from a security camera present on the LAN which are offered via a password protected web portal (.htaccess style username/password) via this DL manager.

This used to work pre-v85 and then stopped working

Files that fail to download with the DL manager can be saved manually using "Save link as..." just fine. Any downloads from the password protected site via the download manager fail consistently. Same issue occurred also with a different download manager. Issue also repro's when download non-video files from another server that uses similar authentication (i.e. router).

See screenshot: first file was a file from a site that was not password protected, second+ ones that failed were all password protected files from the router homepage.

Actual results:

Downloads within DL manager fail instantaneously with "Server error" message when file source is from a password protected website.

Expected results:

Downloads works.

Comment 1

[Tom S [:evilpie]]

Would you feel comfortable trying to use https://mozilla.github.io/mozregression/ to find the regression?

Comment 2

[Andreas Schiffler]

Can't install/use mozregression on Windows due to https://bugzilla.mozilla.org/show_bug.cgi?id=1647533

Comment 3

[Andreas Schiffler]

I can confirm that this is a regression introduced between version 84.0.2 and 85.0.

Steps to reproduce:

• download fails on 85.0
• download 84.0.2 installer 64bit windows
• install
• launch and create new profile
• shutdown and copy old profile data into new profile folder
• launch "C:\Program Files\Mozilla Firefox\firefox.exe" --allow-downgrade
• check that download succeeds on 84.0.2
• update version (About - restart)
• download fails again on 85.0

Comment 4

[Alex Cornestean]

Hello,

I’ve attempted to reproduce the issue on the latest Nightly (87.0a1/20210131211743), Beta (86.0b4/20210131185630) and Release (85.0/20210118153634) under Windows 10 x64, but without success.

To be more specific about what actions I’ve performed, I accessed my wireless modem settings page (which is password protected of course) and tried to download all media from the page using the add-on. All files have been downloaded successfully, without any errors occurring.

If in any case this was not a good scenario through which the issue can be reproduced, please let me know. Maybe point me to some web pages which have the necessary characteristics which will enable me to replicate the bug. I'll also keep investigating and post any results here.

Thank you !

Comment 5

[Andreas Schiffler]

OK - here are my repro steps:

host - Raspberry PI 3B
host OS - Linux raspberrypi 5.4.83-v7+ #1379 SMP Mon Dec 14 13:08:57 GMT 2020 armv7l

sudo bash
--> work as root
apt install apache2 -y
--> installs ver 2.4.38
cd /var/www/html
mkdir test
cd test
htpasswd -c .htpasswd test
--> enter "test" as password
apt-get install joe
--> use "joe" as editor
joe .htaccess
--> add
AuthUserFile /var/www/html/test/.htpasswd
AuthType Basic
AuthName "Protected Folder"
Require valid-user
joe /etc/apache2/sites-available/000-default.conf
--> add
<Directory /var/www/html/test/>
AllowOverride All
Options Indexes FollowSymLinks
Require all granted
</Directory>
service apache2 restart
wget https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4
joe index.html
--> add
Test Page<p/>
<a href="sample-mp4-file.mp4">sample</a><p/>
ip a
--> get IP of server on LAN
--> on Windows device, open Firefox and navigate to http://[IP]/test
--> enter test/test to authenticate
--> right click on page, and select DownThemAll - DownThemAll - video file

Expected: works
Actual: fails with "Server Error"

Comment 6

[Alex Cornestean]

Hello,

I’ve tried to reproduce on Windows x64, based on the info you provided but I don’t quite understand some aspects:

1. In http://[IP]/test does the IP part refer to the public IP of my machine or something else? Cause I tried accessing the URL with my IP and it didn’t work.
2. Do I require a server to be set up and the IP part from the above URL is the IP of the server?

Comment 7

[Andreas Schiffler]

Yes, as per my repro steps above, a web server was setup on the local LAN. It is not public. That was done to simulate my initial error setup of a security cam on the local LAN as close as possible, and make sure the errpr is not caused by the security camera's webserver software (some cheap Chinese make several years old). Once the local webserver is running, it should serve the "Apache2 Debian Default Page" on on http://[IP]. One should be able to create such a repro setup also with a VM (VirtualBox running a Debian installation) or a WSL installation (see https://docs.microsoft.com/en-us/windows/wsl/install-win10). The issue might also repro with a public (non-LAN) website or IP, but I have not tested that. If you provide me with a password protected URL (AuthType Basic) hosting a video file I can try to repro. Also, are there any steps to enable more extensive logging?

Comment 8

[Tom S [:evilpie]]

This is caused by enabling network partitioning on release in bug 1681330.

I bisected this all the way to back to
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=f667f941e5b8e93df11124d8f42cfcac9a8c4f2b&tochange=fab7c4f54054ceb06504c7ddac380858e2521fc4
Bug 1643288 - Isolate HTTP channel authentication per first-party when privacy.partition.network_state is set to true

Comment 9

[Tom S [:evilpie]]

I tested this like this:

• Goto https://authenticationtest.com/HTTPAuth/
• Login with "user" and "pass"
• Install DownThemAll!
• Click on the extenion button and "Add Download"
• Download "https://authenticationtest.com/HTTPAuth/" again

Comment 10

[Tomislav Jovanovic :zombie]

Hey Andrea, this seems it breaks any downloads with HTTPAuth from extensions. Do you have an idea how we might fix this?

Comment 11

[Rob Wu [:robwu]]

Bug 1685862 is another bug where a fix is to create a new content principal based on the download URL instead of the moz-extension:-URL (triggeringPrincipal).

If this bug is due to an incorrect triggeringPrincipal (it's moz-extension:, not the download URL nor the system principal), then using a content principal based on the download URL may fix this bug.

Comment 12

[Rob Wu [:robwu]]

Is the issue still reproducible in Firefox 89? For common cases, this was addressed by bug 1691907.

Comment 13

[Andreas Schiffler]

(In reply to Rob Wu [:robwu] from comment #12)

Is the issue still reproducible in Firefox 89? For common cases, this was addressed by bug 1691907.

Issue does not repro anymore in version 89. Thanks!

Comment 14

[Rob Wu [:robwu]]

Thanks for your confirmation!