Closed Bug 1891321 Opened 2 years ago Closed 2 years ago

(security) Firefox executes JavaScript that is embedded in HTML code masquerading as a PDF document

Categories

(Firefox :: File Handling, defect)

Product:
Firefox
Component:
File Handling
Type:
defect

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: mozilla.bugs.grokchem, Unassigned)

Details

Attachments

(1 file)

Van-Bommel-Menukaart-November-2023-FOOD_edited.pdf.html
1.52 KB, text/html
Details

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36

Steps to reproduce:

Visited this page: https://cafevanbommel.nl/menu/

Noticed this “PDF” menu: https://cafevanbommel.nl/wp-content/uploads/2023/11/Van-Bommel-Menukaart-November-2023-FOOD.pdf

Since I am on a measured rate network, it’s important that I see the size of a file before fetching it, particularly with a PDF menu because with food pics menus can get quite big. So I ran this:

$ torsocks curl -LI 'https://cafevanbommel.nl/wp-content/uploads/2023/11/Van-Bommel-Menukaart-November-2023-FOOD.pdf'

Output showed that the “PDF” file was only ~1k:

HTTP/2 200 
date: Tue, 09 Apr 2024 16:01:40 GMT
content-length: 1480
cache-control: no-cache, no-store, must-revalidate, max-age=0
cache-control: no-store, max-age=0
server: imunify360-webshield/1.21

So I fetched it with wget. It’s a fake! It’s actually an application! HTML-wrapped JavaScript is pretending to be a PDF doc. If I run that application to fetch the PDF I was deceived into thinking I was downloading originally, I have no way to first see how big the file is. So I had to go to the library to fetch the file just in case it’s really big.

Actual results:

Firefox 102.6.0esr fetched a file with a “PDF” extension that instead of containing PDF content actually contained JavaScript within HTML code. Firefox took the liberty of executing the JavaScript which ultimately fetched the real menu and rendered it, as unwitting users might expect.

Expected results:

Firefox should have either fed the “PDF” to pdf.js (which would correctly choke on it), or alternatively FF should have detected that the “PDF” is not really a PDF and warn the user. It should not have arbitrarily executed the JavaScript therein.

I mention security in the title because users are being deceived into executing a JavaScript application when in fact they think they are simply fetching a PDF. There should be a security review on this. But I’m not ticking the “security” box because any JavaScript pushed by the website would execute anyway, even without the use of a deceptive instrument. So this disclosure will not result in malice not otherwise possible.

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Core & HTML' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → DOM: Core & HTML
Product: Firefox → Core
Version: other → unspecified

It’s perhaps worth noting alternative “browsers” that do the right thing and simply fetch the the fake PDF without executing the js inside:

  • wget
  • Android’s Navi (download manager)
    But yeah, they’re getting lucky because they wouldn’t know what to do with JavaScript anyway.
Component: DOM: Core & HTML → File Handling
Product: Core → Firefox

The severity field is not set for this bug.
:Gijs, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(gijskruitbosch+bugs)

Unfortunately I'm struggling to do much here. Loading the URL in comment 0 ( https://cafevanbommel.nl/wp-content/uploads/2023/11/Van-Bommel-Menukaart-November-2023-FOOD.pdf ), I only ever see a ~1.66mb pdf. Same with curl as with Firefox. No 1KiB HTML file with script or anything like that. So I can't investigate much or consider what changes Firefox should make.

I expect that your request with torsocks (unsure what that is...) was hitting some kind of firewall (which is made more plausible given the server header says it's "webshield"), which changed the response you received. It's unclear to me based on your report whether Firefox hit the same page or was "let through" to the "real" HTML/server immediately. If you have a copy of the HTML and script that would help.

(In reply to ed from comment #0)

HTTP/2 200 
date: Tue, 09 Apr 2024 16:01:40 GMT
content-length: 1480
cache-control: no-cache, no-store, must-revalidate, max-age=0
cache-control: no-store, max-age=0
server: imunify360-webshield/1.21

Note that there is no content type here, so it's completely unclear to the browser what type of response it's receiving. You think it's a PDF based on the .pdf in the URL, but Firefox (and other browsers) will probably sniff bytes in the response to determine what it is, unless the server explicitly sends a no-sniff header (docs) or mime sniffing is otherwise disabled.

Firefox 102.6.0esr fetched a file with a “PDF” extension that instead of containing PDF content actually contained JavaScript within HTML code. Firefox took the liberty of executing the JavaScript

At what point did/does this happen? That is, is the pdf-that-is-really-HTML downloaded to your machine and saved to disk, and then executed from file: once you open it, or is it loaded directly from cafevanbommel.nl and run on that origin?

Firefox should have either fed the “PDF” to pdf.js (which would correctly choke on it), or alternatively FF should have detected that the “PDF” is not really a PDF and warn the user. It should not have arbitrarily executed the JavaScript therein.

On what basis? Purely the fact that the URL ends in .pdf? That likely wouldn't be web-compatible... and might lead to other security problems for sites that permit file uploads.

(In reply to ed from comment #2)

It’s perhaps worth noting alternative “browsers” that do the right thing and simply fetch the the fake PDF without executing the js inside:

  • wget
  • Android’s Navi (download manager)
    But yeah, they’re getting lucky because they wouldn’t know what to do with JavaScript anyway.

Did you try chromium or webkit in any guise?

Flags: needinfo?(gijskruitbosch+bugs) → needinfo?(mozilla.bugs.grokchem)

I only ever see a ~1.66mb pdf. Same with curl as with Firefox. No 1KiB HTML file with script or anything like that. So I can't investigate much or consider what changes Firefox should make.

Apparently there is a lack of network neutrality situation. The web server is discriminating against people on the basis of network, giving different treatment to different people depending on IP address, creating an access inequality. Reproducing the problem would seem to require using a shared network (VPN, Tor, public library, CGNAT IP address, etc). Tor would likely be the most reliable reproducer.

If you have a copy of the HTML and script that would help.

I will attached it.

At what point did/does this happen? That is, is the pdf-that-is-really-HTML downloaded to your machine and saved to disk, and then executed from file: once you open it, or is it loaded directly from cafevanbommel.nl and run on that origin?

It’s automatic. I do not have to explicitly open the file. And that’s the problem. As a separate test, I disabled js:

about:config → javascript.enabled → false

Then when I visit the “PDF” (over Tor), the window simply says “Please wait while your request is being verified...”, and from there it does nothing forever. If I enable js, then my noscript plugin blocks the js from running, and it’s the same result. If noscript is configured to trust cafevanbommel.nl, then it executes the js willy nilly and launches pdf.js automatically.

Did you try chromium or webkit in any guise?

Chromium behaves the same way. The user is deceived into believing they are simply doing an HTTPGET on a PDF, when in fact it’s some sneaky javascript that executes without the user’s knowledge. This mechanism could likely be used for more malicious acts, such as a script that port scans their LAN, for example. For the case at hand, it breaks people’s ability to first examine the HTTP headers for the document size in advance before fetching it.

The fact that Chromium and Firefox both support this deception (however inadvertent), it makes these two browsers an enabler for shenanigans. As someone on a limited network, I always run torsocks curl -LI $url on binary content before fetching to first know whether the download will suck dry the credit and cost me money. Generally this behavior impacts poor people who cannot afford unlimited internet service. In any case, a well-behaved browser would /minimally/ give a pop-up warning that says (maybe in so many words) “warning: you thought you were doing an HTTPGET on a PDF document, but what arrived was actually a JavaScript application instead. Do you want to execute this app?” I think if users were properly informed, that would fix the problem well enough from the browser standpoint, noting of course that browsers are supposed to serve the best interests of the users, not the website administrators.

Flags: needinfo?(mozilla.bugs.grokchem)

Attached is the JavaScript sent by the server when a Tor user requests fetching a PDF. There was a unique ID of some kind in the file, which was redacted (hence “_edited” in the filename).

It’s worth noting that emacs expects this JavaScript to be a PDF, and tries to convert it from PDF to PNG. This showcases the transparency problem that arises out of the abuse of this webserver to use a PDF extension for a JavaScript application. Users should be able to inspect the JavaScript before running it and the shenanigans in play thwart that.

Attachment #9402242 - Attachment description: Van-Bommel-Menukaart-November-2023-FOOD_edited.pdf → Van-Bommel-Menukaart-November-2023-FOOD_edited.pdf.html
Attachment #9402242 - Attachment mime type: application/pdf → text/html

(In reply to ed from comment #5)

If you have a copy of the HTML and script that would help.

I will attached it.

Thank you.

At what point did/does this happen? That is, is the pdf-that-is-really-HTML downloaded to your machine and saved to disk, and then executed from file: once you open it, or is it loaded directly from cafevanbommel.nl and run on that origin?

It’s automatic. I do not have to explicitly open the file. And that’s the problem. As a separate test, I disabled js:

about:config → javascript.enabled → false

Then when I visit the “PDF” (over Tor), the window simply says “Please wait while your request is being verified...”, and from there it does nothing forever. If I enable js, then my noscript plugin blocks the js from running, and it’s the same result. If noscript is configured to trust cafevanbommel.nl, then it executes the js willy nilly and launches pdf.js automatically.

OK. What I guess I should have asked is "when the JS runs, what does the URL bar say - does it start with https://cafevanbommel.nl or similar, or with file:///... ". Based on what you're saying, it sounds like it's the former. That's good - at least it's not running script from a file on your local machine.

Did you try chromium or webkit in any guise?

Chromium behaves the same way. The user is deceived into believing they are simply doing an HTTPGET on a PDF, when in fact it’s some sneaky javascript that executes without the user’s knowledge.

So, I hate to be That Person, but this is unfortunately how HTTP works:

  1. the url path portion provides no guarantee of the mimetype of the content the server returns. There is no specification or technical measure that prevents a server from returning a PDF when accessing https://example.com/somewhere/, or https://example.com/somewhere.html or https://example.com/somewhere.exe or https://example.com/somewhere.pdf or anything like that. And, inversely, for it to return HTML in the same cases.
  2. there is nothing that obliges the server to return the same thing for 2 identical requests. That could be as benign as a POST request for a server-side search returning different results (despite identical input - just because server state has changed in the interim), a visitor counter for a GET request, or randomly-generated compliment/poem/whatever - or malicious, or DRM/content-control... it's up to the server. There are some conventions that GET requests should not manipulate server data (REST-ful APIs), but they are conventions and there is no way to technically enforce them. As the browser cannot see or manipulate the server's internal state, there is nothing it can do about this.
  3. a page that is allowed to run JS can then redirect to some other page (PDF or HTML; of any size). Even a page that isn't allowed to run JS can do so via HTML meta refresh mechanisms (on top of the HTTP ones that curl or wget would have resolved in the torsocks case, depending on what options you pass).

This mechanism could likely be used for more malicious acts, such as a script that port scans their LAN, for example. For the case at hand, it breaks people’s ability to first examine the HTTP headers for the document size in advance before fetching it.

Per the above, there isn't really anything the client can do about this.

The fact that Chromium and Firefox both support this deception (however inadvertent), it makes these two browsers an enabler for shenanigans. As someone on a limited network, I always run torsocks curl -LI $url on binary content before fetching to first know whether the download will suck dry the credit and cost me money. Generally this behavior impacts poor people who cannot afford unlimited internet service.

I'm very sympathetic to the argument that the choices of the server here disproportionately impact people for whom bandwidth is a concern.

In any case, a well-behaved browser would /minimally/ give a pop-up warning that says (maybe in so many words) “warning: you thought you were doing an HTTPGET on a PDF document, but what arrived was actually a JavaScript application instead. Do you want to execute this app?”

I think this gets a bit confused here. The bandwidth concern seems orthogonal to the PDF-ness. That is, for an HTML response (even for .html URLs), it is quite possible that the server returns either a small or large response depending on its whims. If the webshield thingy ended up redirecting a huge HTML file that was the same size as the PDF, the bandwidth bill would have been the same, right?

Inasmuch as the .pdf thing should trip such a warning, this gets very messy. Most users would not be interested in this - even in the reported case, a PDF does eventually arrive. Similarly .exe files can be both downloads and in some cases server-run scripts which produce HTML. So it's a very error-prone heuristic, were we to implement it, with limited user value - and so I don't think it's a good idea.

I think if users were properly informed, that would fix the problem well enough from the browser standpoint, noting of course that browsers are supposed to serve the best interests of the users, not the website administrators.

Indeed, but I think the PDF/HTML-ness of a given response is a red herring here. I think a proper solution would be some kind of bandwidth limiting that is built-in to the browser, to cater for folks on limited connections, so that at the point where the small script page redirects to a huge second thing (PDF or otherwise), the browser intervenes before loading the whole thing.

We don't have good tools for this in Firefox today. Given your use of torsocks, I guess you are technically minded and could try doing this via the browser toolbox ( https://firefox-source-docs.mozilla.org/devtools-user/browser_toolbox/index.html ) and limiting the network speed there. Then when something takes a significant amount of time to load, you have time to inspect the response headers and decide what to do. Of course, this is neither ergonomic nor without side effects (things will be slower!). A webextension should be able to do better (by inspecting Content-Length and refusing to load anything over X KiB), but I can't easily find something ready-made for this.

Anyway, I don't think the bandwidth concern is related to the PDF-ness here.

What I guess I should have asked is "when the JS runs, what does the URL bar say - does it start with https://cafevanbommel.nl or similar, or with file:///... ".

The URL prefix does not change.

url path portion provides no guarantee of the mimetype of the content the server returns.
there is nothing that obliges the server to return the same thing for 2 identical requests.

I have a problem with the deception. You seem to be saying the standards do not force web servers to be ethical or well-behaved. I have no reason to say the contrary, but the question really is: are browsers forced to play along with the deception? Or can a smart browser say: shenanigans detected -- we should inform the user, without the browser violating a standard?

I think this gets a bit confused here. The bandwidth concern seems orthogonal to the PDF-ness.

The problems are related by cause and effect. I wouldn’t step outside of Tor but if I were to inspect the headers of the PDF URL on a clearnet connection, I believe the content-length field would be honest in that case. The server’s deceptive shenanigans have broken that possibility in a way that discriminates against poor people who also happen to use Tor. The means of breakage is a deceptive practice that I reported as a bug. But indeed the bug (which is that the browser is complicant in the deception) does not necessarily trigger the particular harm that I experienced. The deception can cause harm in various ways and what I experienced was just an example of one harm that comes of it.

It’s because the two most mainstream browsers accept the deception that enables the deception to carry on. If the browser were to be diligent and give users a pop-up warning, the web devs would have opted not to use the deceptive mechanism in the first place. The browser’s acceptance reinforced their deployment of an enshitified resource.

If the webshield thingy ended up redirecting a huge HTML file that was the same size as the PDF, the bandwidth bill would have been the same, right?

If huge HTML were a problem that those of us on measured rate connections had to contend with, then we would also have to look at content-length headers on all html objects. Luckily that’s never the case (so far). In 2010 the avg size of a webpage was 500kb. in 2024 the avg size went up to 2mb. That would include everything needed to render a webpage. Studies show that a majority of that is images. So those on measured rate connections solve that problem by setting permissions.default.image to 2. An abusive/reckless website could embed images in the HTML in base64. It seems uncommon but if that were to become a thing, then we would need a browser that gives the content length of every HTML fetch before clicking.

ATM, we have the option of doing a content-length check on every object that we expect to be large (assuming no deception is in play). Some sneaky servers withhold that info (especially if it’s a tracker pixel in which case they want to conceal the fact that the image is so tiny it could only be used as a tracker). Nonetheless, the JS masquerading as a PDF problem inherently masks the size of the PDF from the user under deception.

Most users would not be interested in this - even in the reported case, a PDF does eventually arrive.

It’s hard to speak for most users but most users are not interested in security. They might rather be exploited in some way than to have a browsing session with warnings. But this is exactly why web designers would then face pressure to not use deceptive design patterns. The browser has an ethical duty to inform and protect it’s user. If a diligent browser warns a user about shenanigans, the website would rightfully risk losing traffic. A restaurant site does not want to send a signal to users that they run a dodgy website.

I think a proper solution would be some kind of bandwidth limiting that is built-in to the browser, to cater for folks on limited connections, so that at the point where the small script page redirects to a huge second thing (PDF or otherwise), the browser intervenes before loading the whole thing.

Yes, that would be a good solution to the bandwidth protection issue. It would also be good for the environment because even when I’m on an unlimited connection I still disable images because most of them are just an abuse of cloud energy. But it needs to be clear the problem underlying bug 1891321 is that the browser facilitates deception. The side-effect of fetch size masking is not what the bug report is about, just an example of harm done by the deception.

You can treat the root cause of the problem, which is to create a mechanism to push back against deception, or you can treat one of (perhaps many) symptoms, and have a browser that’s mindful of bandwith waste (which ideally would give both an object size indicator as well as a CO₂ estimate). Or both. Make it so Firefox insists on honesty and also a bandwidth control feature would be useful.

It looks like no security folks have reviewed this bug.

(In reply to ed from comment #9)

It looks like no security folks have reviewed this bug.

I mean, I kind of resent the implication that I don't know what I'm talking about, but I can ask for a second opinion.

Flags: needinfo?(tom)
Flags: needinfo?(dveditz)

I agree that this is not a security issue; but rather a mismatch between your expectation and how the web works. 90%* of the time, it works as expected, because norms are norms and they're what normally happens. But 10%* of the time the web behaves differently from what you're expecting - either because the extension doesn't match the delivered mime type, or the server has discriminated against you based on your network (Tor) or your privacy settings (fingerprinting protections). Both of which I am sympathetic to as both a Tor developer and the primary maintainer of Tor's Resist Fingerprinting feature. And we dutifully log the latter so we can potentially improve the feature - I encounter Please wait while your request is being verified... all the time (well, the one I see the most is We want to make sure it is actually you we are dealing with and not a robot.)

* A made up number; but even if it were 99%/1% of the time; 1% is still too frequent for us to break compatibility with 2 million websites.

Saying that we should provide more friction to users because the extension doesn't match the returned content is making a big bet that website owners will change their behavior because their visitors are being inconvenienced. I tend to think the opposite, that this would push people away from Firefox. And also that it wouldn't even be understood by a large portion of our audience - the most common Operating System has hidden file extensions by default for over 20 years. I'll also note that this bet is only applicable when the website is trying to attract visitors; which is mostly limited to shops and news sites - not services like your Gas/Electric/Bank. If you don't use their websites... they don't particularly care. Which has been borne out by what sectors of the web are the hardest to get to update when we are trying to close loopholes and make the web more secure.

The notion that Firefox should provide more control to users with metered connections is orthogonal to the assertion that this is a security concern. For what it's worth, Tor includes non-trivial bandwidth overhead; so use it if you want to for the privacy, but know that other solutions (like a VPN) would be more bandwidth efficient.

Flags: needinfo?(tom)

I mean, I kind of resent the implication that I don't know what I'm talking about, but I can ask for a second opinion.

Ah, sorry, that was not my intent. I didn’t realize you had a security background. I noticed that on security-relevant bugs the bug tracking UI shows with some kind of tag that security staff have reviewed the reports. I did not see that indicator. Perhaps the bug tracker should be configured so when you review something it gets that tag.

It’s perhaps worth noting alternative “browsers” that do the right thing and simply fetch the the fake PDF without executing the js inside:

Those are not browsers, those are HTTP utilities. A "browser" is an HTML document viewer that knows how to request HTTP resources (at a minimum; of course people expect them to handle lots more technologies these days).

The browser’s acceptance reinforced their deployment of an enshitified resource.

"enshitified" as coined by Doctorow refers to a specific pattern of abuse in the way some services and products evolve, particularly those funded by venture capital. It is NOT a synonym for "shitty"—we already have that word and if that's all it meant Doctorow would have used it or any number of other words instead. The more it is misused as the hot new "crappy" the faster we will lose it as a tool to expose this specific pattern of abuse.

The fact that Chromium and Firefox both support this deception (however inadvertent), it makes these two browsers an enabler for shenanigans.

...

It’s because the two most mainstream browsers accept the deception that enables the deception to carry on. If the browser were to be diligent and give users a pop-up warning, the web devs would have opted not to use the deceptive mechanism in the first place.

I know you are upset by this practice and I understand your bandwidth concerns, but have you taken a step back to think about the fact that the website is worried that you are the deceptive one? Not you personally, but your Tor-node origin unfortunately makes you indistinguishable from the bots and people who use Tor traffic to hide the source of their abuse. You're worried about the cost of downloading one huge PDF; the web site is worried about the cost of sending thousands of huge PDFs to bots who are trying to run up their bills or take them offline through too much traffic. I doubt this Dutch café itself is even aware their web site is doing this to you, but rather this is more likely something done automatically by their hosting provider, maybe in the data center's web application firewall (WAF) before the request even gets to the real café site.

This "deception" isn't because Chrome and Firefox allow it. All browsers will "allow" it because that's what the HTTP and HTML specs say to do in this situation. I'd even argue the URL isn't deceptive at all. The URL spelling is irrelevant to the specifications and the browser, but it's meant as a clue to a human, and if you click on that .PDF link you do (eventually) get a .PDF out of it. A "normal" link could redirect a number of times, perhaps to a CDN, before you get a response. You can think of this interstitial page as a "slow redirect". It frustrates your personal strategy for checking out the size of a file before downloading it, but that's an unintentional side-effect rather than "deception".

As framed, this bug report is invalid. It would be better to start a new bug that centers your desire for Firefox to help you limit your bandwidth consumption. Maybe we could have a "data saver" mode that let you specify the maximum file size you'd allow.

Flags: needinfo?(dveditz)

I don't understand how that interstitial page proves you aren't a bot though. The heart of it is adding two JSFuck-encoded numbers together and submitting that back to the site.

  • What does that prove, that I've got a working JS interpreter? Do bots not process JS?
  • What value does the obfuscation serve? If you do have a working JS interpreter it's trivial to get the result with or without the obfuscation.

Presumably you'd get different numbers each time you encounter the interstitial so bots can't just memorize the values.

It’s perhaps worth noting alternative “browsers” that do the right thing and simply fetch the the fake PDF without executing the js inside:

Those are not browsers, those are HTTP utilities. … "enshitified" as coined by Doctorow refers to a specific pattern

Okay, but let’s not clutter the discussion with esoteric semantics. If something I’ve said is genuinely not understood, then I’m happy to clarify. It’s a complex topic with lots of moving parts and more interesting factors to unpack to justify much time on language refinement.

A "browser" is an HTML document viewer that knows how to request HTTP resources (at a minimum; of course people expect them to handle lots more technologies these days).

I hate to deviate from what I just said, but exceptionally this definition matters in the case at hand (but fwiw, it proves your point even more that cURL et al is not a “browser”). A post-90s browser is not just a HTML doc viewer -- it’s also an application execution platform/engine (JavaScript). Users’ ability to distinguish a “document” from an “app” has been lost over the past ~20 yrs. And especially in the case at hand the user is explicitly led to believe they are fetching a document when in fact it’s an application. The web has inherently blurred this line even without proactive deception, sadly enough, amid web designers proactively misleading users. I could write a book on the harm done by blurring the doc/app line, but hopefully not in this thread.

I know you are upset by this practice and I understand your bandwidth concerns, but have you taken a step back to think about the fact that the website is worried that you are the deceptive one?

I have to say this is both inaccurate and irrelevant. The Tor community is not using deception in the slightest. The Tor Project is wholly transparent. Every exit node is published, as well as the code. Tor Project bends over backwards to be transparent to the full extent possible.

It’s also irrelevant because the browser’s job is to serve the user (not the web server). Firefox is also Tor-unaware, simply using a proxy specified by the user. Firefox has no awareness of what network it’s on. So even if you could describe a network as “deceptive” (e.g. if the Tor network were to actually deceive outside hosts in some way), it’s beyond the scope of Firefox’s perview and control.

Not you personally, but your Tor-node origin unfortunately makes you indistinguishable from the bots and people who use Tor traffic to hide the source of their abuse.

Masquerading under a different identity would be deceptive but exit nodes are public and designated as shared IPs. There is no deception inherent in the Tor network. There are also beneficial bots, so I also have to object to any inference that suggests bots are grouped together and wholly regarded as abusive. The abuse to speak of here is that inherent in blunt grouping of bad actors with legit users by those unconcerned with colatteral damage and collective punishment.

It’s important to point out that Immunify360 is unlikely only targeting the Tor community for unfavorable treatment. Services like Immunify360 (e.g. Cloudflare) discriminate against many demographics of people and Tor is only one of the marginalised groups. We cannot know the full extent of people marginalised due to their non-transparency, but for example we know from experimentation that Cloudflare gives harsh treatment to users on a CGNAT network. Impoverished communities cannot afford an IPv4 address for everyone so ISPs share IPs from a pool of IPs. CF uses a blunt collective punishment against people on CGNAT networks without their knowledge or control. Blind people have no need to fetch images, which makes their traffic look more like that of a bot, and so they face hostile treatment as well. Users of public libraries also experience unfair treatment because those are also shared IPs. As are commercial VPNs.

The problem the report spotlights is independent of the server discriminating against different demographics of people. So the fixation with Tor is a red herring. A web server can just as well use the same deception against all visitors. Tor is only relevent here as a reproducer for someone working on this bug.

You're worried about the cost of downloading one huge PDF; the web site is worried about the cost of sending thousands of huge PDFs to bots who are trying to run up their bills or take them offline through too much traffic. I doubt this Dutch café itself is even aware their web site is doing this to you, but rather this is more likely something done automatically by their hosting provider, maybe in the data center's web application firewall (WAF) before the request even gets to the real café site.

Not our problem. Web admins have varying degrees of competency. Some admins are good at providing service without discriminating against whole demographics of people. They do not rely on crude IP reputation techniques. It’s not Mozilla’s mission to make security easy for service providers, most particularly if the server’s security policy undermines access equality and ultimately Mozilla’s own manifesto.

This "deception" isn't because Chrome and Firefox allow it.

The cause of the deception is not borne from the browser, but the browser is still a cause in the sense that the browser facilitates the deception, thus enabling it. We cannot blame the browser for a web designer deciding to invent a deceptive mechanism, but the browser can of course be blamed for any willful enabling of deception once it’s known.

All browsers will "allow" it because that's what the HTTP and HTML specs say to do in this situation. I'd even argue the URL isn't deceptive at all. The URL spelling is irrelevant to the specifications and the browser, but it's meant as a clue to a human, and if you click on that .

The specs are the wrong source to use for evaluating deception. Deception is determined by human psychology. The specs are (at best) a product of engineering that may or may not have taken good account of human factors. And judging from the state of the web today, some would say the specs have failed in that regard. Certainly the specs are not a good source for evaluating the transparency of an implementation.

PDF link you do (eventually) get a .PDF out of it. A "normal" link could redirect a number of times, perhaps to a CDN, before you get a response. You can think of this interstitial page as a "slow redirect". It frustrates your personal strategy for checking out the size of a file before downloading it, but that's an unintentional side-effect rather than "deception".

HTTP redirections are very different from execution of an application. Superficially, I see no case for inherently calling an HTTP redirect “deceptive”. But calling a JavaScript program a “PDF document” is patently false and clearly deceptive. Most importantly, I was fooled. I saw a ~1k PDF doc and said: sure, I’ll download that. If I had known that it was actually a JavaScript app, I would not have downloaded it. I was deceived and the deception influenced my behavior. If we were talking about a simple redirect, not an app masquerading as a doc, I still have full transparency and control in that case. I can check whether a redirect is in play and I can also control whether redirects are followed (certainly in curl, not sure about ff).

It’s also worth noting this deception has a couple different outcomes. If curl -LI $pdf shows the filesize is tolerable, I may or may not use wget to grab it. Two scenarios:

① If I use wget, then I get screwed out of credit but just to the extent of the code size. Most users at that point would do evince $local_pdf_file and evince says: get lost.. this isn’t a PDF. Then they use file to find out it’s ascii, then they cat it. The loss is some of the user’s time and a small wasted download.

② I sometimes go straight to FF. cURL says the PDF is small but if it’s not a PDF I’m likely to want saved I’ll go straight to the FF window (where the URL was copied from originally) and click it. Then I see the download indicator taking a long time and rush to kill it. At which point I realized I just got burnt out of significant credit because of the deception. The 1k doc might turn out to be 100mb, for example. The bug effectively costs users money in this case. Some carriers charge 50¢/mb.

As framed, this bug report is invalid. It would be better to start a new bug that centers your desire for Firefox to help you limit your bandwidth consumption. Maybe we could have a "data saver" mode that let you specify the maximum file size you'd allow.

The harm caused by the bug can indeed by remedied with an enhancement. But that’s insufficient because (what I will call) the thesis bug (the bug of deception this report is about) still persists even if users have more bandwidth consumption control. You fix one symptom but users are still manipulated into conflating a doc for an app.

This enhancement request covers the request to give users bandwidth consumption control. While bug 1891321 requests enabling users to be informed when an app masquerades as a document.

I tend to think the opposite, that this would push people away from Firefox.

This really depends on how the transparency is given. You probably envisioned a giant pop-up overtaking the screen when you said that.

A clever fix would avoid the problem you describe. Imagine a user clicks on an app masquerading as a fake “PDF”. The browser would have to fetch the object because it could not know¹ of the masquerade before hand. But from there, the browser has choices. It could have a bar the size of the bookmark bar unfold and say something like “your PDF is actually an application; hit ESC to stop it or wait 5 seconds for it to execute.”

The notion that Firefox should provide more control to users with metered connections is orthogonal to the assertion that this is a security concern.

They are certainly two separate change requests. But not orthogonal (which implies no relationship) because one bug is exacerbated by the other bug. If users had bandwidth consumption control, then the bug of deception enabling would impact fewer people in different ways. And if the deception facility were fixed, then users could escape getting burnt on a big download even if they don’t have any extra bandwidth controls.

¹ Actually a very clever browser might know that a PDF would unlikely be 1k in size, in which case it could even pre-empt the fetch with info to the user. But I’d say that degree of advancement is far out on the horizon.

HTTP redirections are very different from execution of an application. Superficially, I see no case for inherently calling an HTTP redirect “deceptive”.

Dan didn't say "HTTP". Redirections are commonly done via other mechanisms, which I already mentioned in my earlier comment.

The cause of the deception is not borne from the browser, but the browser is still a cause in the sense that the browser facilitates the deception, thus enabling it.

"Deception" requires intent to deceive. I am not convinced that this even exists - a WAF or other mechanism inserting itself is not intended to be deceiving you, it intends to protect the web server. The consequence might be that you feel deceived. That is not the same.

We cannot blame the browser for a web designer deciding to invent a deceptive mechanism, but the browser can of course be blamed for any willful enabling of deception once it’s known.

Only inasmuch as the browser is in a position to recognize such intent to deceive. I do not think it is.

The mismatch of file extensions with response content types is common on the web (index.php is the trivial example, but cgi-bin/request.exe and various other forms exist). File sizes are a proxy and not a very good one at that - should the browser block 2kB "PDF"s that are HTML? What about 10kB? LaTeX PDFs are pretty small... - there is no technical reason that the filesize would really be correlated with the true content type, and even if we invented a standard for this, if we're really hypothesizing that the intent to deceive (or for malice generally) is there, web servers would respond by simply adding junk into their page to up the content size, or produce an outsized Content-Length header (which many websites already do because servers are buggy and compression is somehow harder than one would think in the year 2024, but I digress).

[after fetching the HTML from a .pdf URL the browser] could have a bar the size of the bookmark bar unfold and say something like “your PDF is actually an application; hit ESC to stop it or wait 5 seconds for it to execute.”

Tom already responded in comment 11, but just to spell it out, here are the consequences I believe this would have, in order of likelihood:

  1. Firefox would break websites and the relevant specs. It's unclear at what point we would decide not to "execute". Does it depend on the presence of JS or not? Because of how HTML works, we cannot even parse the full document if script is enabled-but-somehow-not-running (because results of document.write and similar APIs alter the resulting node tree). Does a plain HTML doc with a <meta refresh header count as an "application"?
  2. People would ignore the warning.
  3. People would be confused because they clicked something and then got a blank page (we can't render anything...) with a bar. By the time they realize what is happening, the bar disappears, likely before they had time to read it.
  4. People realize this is just one more thing where software gets in their way and they accept that when they use certain sites they have to wait 5 seconds every time.
  5. People complain to us about this needless interruption in tools they use every day, and/or switch browsers.
  6. Web site owners alter their sites to circumvent the browser logic (but without changing their practice of "protecting" large resources), or complain to us about cases where this happens on their website and they deem the warning to be overly scary or interruptive to their business, and argue that for whatever reason their usecase is not intended to be deceptive.

Fundamentally your idea is halfway between having the browser be an active User Agent and make a decision (namely, not to execute the JS immediately), but also delegating the actual decision to the user, but without providing the user meaningful information or opportunity to make the decision (we and the user cannot know what the "application" does once executed - all we know is it's in our way when we expected to just see a PDF).

This is a bad idea and we will not do it. I do not think there is any practical measure we can take here that would have the results you desire.

Status: UNCONFIRMED → RESOLVED
Closed: 2 years ago
Resolution: --- → WONTFIX

Dan didn't say "HTTP". Redirections are commonly done via other mechanisms, which I already mentioned in my earlier comment.

Not the way he described it. Dan described an HTTP redirect. Conflating an application with a document is not mere “redirection”. If he actually meant something other than HTTP redirect then he used the wrong language to articulate his thought.

"Deception" requires intent to deceive. I am not convinced that this even exists - a WAF or other mechanism inserting itself is not intended to be deceiving you,

It is intentional. It’s an application being called a PDF document. A non-deceptive way to name the file is menu_downloader.js, not menu_….pdf.

it intends to protect the web server. The consequence might be that you feel deceived.

To imply that “intent to protect the webserver” inherently obviates intent to deceive is a false dichotomy. It’s deception whether or not it protects the webserver. They can protect the webserver using an honest design and language. Calling the file menu_downloader_app.js is both honest and gives them the protection they are after.

Only inasmuch as the browser is in a position to recognize such intent to deceive. I do not think it is.

I mentioned two possible mechanisms where the browser can make use of the information it has, one before the 1st fetch and one before the 2nd fetch. The latter is absolutely trivially feasible. If the curl and file commands can do it, so can Firefox.

The infeasibility you describe refers to the advanced idea of the browser being smart enough to preempt the first fetch. It’s not science fiction but I believe I made it clear that that would be a quite sophisticated bit of logic for the far future. That is a separate advanced fix and should not cloud or conflate the trivial fix of making the first fetch as blind as the status quo and then concretely seeing that what was fetched masqueraded as something else.

0.… Does a plain HTML doc with a <meta refresh header count as an "application"?

Why would Firefox decide to execute something if it cannot even work out whether it is an app that can be executed? This makes no sense. Is Firefox blindly saying to the JS engine “here, we don’t know if this is an app but if so plz execute it for us”? If that’s the case, there’s a problem with the API between the JS engine and the browser. Firefox should be able to say “don’t execute this, just tell us if it’s an app”. Ultimately the JS engine must know at some level whether it has been passed an application.

  1. People would ignore the warning.

Some people will and some will not. It depends on whether the user wants autonomy or agency.

  1. People would be confused because they clicked something and then got a blank page (we can't render anything...) with a bar. By the time they realize what is happening, the bar disappears, likely before they had time to read it.

It depends on how you implement it. Firefox uses the browser window to inform users all the time. You can use that opportunity or not. Firefox also uses that opportunity to prompt the user with choices. E.g. “cert does not verify… [advanced button]… accept the risk and continue?” There are many ways to solve this bug in a way that empowers users who want agency while automating users who do not. Hands-off users can let it expire and users who click to pause/stop execution can be given further dialog if they want it, and even a chance after reading more info to re-decide.

  1. People realize this is just one more thing where software gets in their way and they accept that when they use certain sites they have to wait 5 seconds every time.

Include a button on that info bar “[never show this warning again]”.

  1. People complain to us about this needless interruption in tools they use every day, and/or switch browsers.

People also rightfully complain when browsers prioritize the needs of the web service above the user. There is an optimum implementation that minimizes complaints, and that optimum is most certainly not in the status quo extreme of silently allowing preventable deception to play out without users having so much as a config control.

  1. Web site owners alter their sites to circumvent the browser logic (but without changing their practice of "protecting" large resources), or complain to us about cases where this happens on their website and they deem the warning to be overly scary or interruptive to their business, and argue that for whatever reason their usecase is not intended to be deceptive.

Mozilla can (and should) tell website owners to pound sand because Mozilla is not serving the webmasters. Mozilla Firefox is expected to serve the browser user, and only the user. This conflict of interest is eyebrow raising. Please note Principle 5 of the Mozilla manifesto:

Individuals must have the ability to shape the internet and their own experiences on it.

Perhaps that’s not as explicit as it should be, but it is not intended to put businesses and webmasters needs above the individual users. The browser should be focused on serving the user.

Fundamentally your idea is halfway between having the browser be an active User Agent and make a decision (namely, not to execute the JS immediately), but also delegating the actual decision to the user, but without providing the user meaningful information or opportunity to make the decision (we and the user cannot know what the "application" does once executed - all we know is it's in our way when we expected to just see a PDF).

This is quite a strawman take of what was proposed as a fix to the problem here. Precisely the problem with the status quo is that users are both disempowered and simultaneously uninformed (in fact deceived). Mozilla can fix this in a way that gives users agency to the extent that they want. Mozilla can also inform the user to the extent that the user wants. This is already somewhat demonstrated when it comes to questionable TLS certs.

This is a bad idea and we will not do it.

The status quo is a bad idea and the perpetuated deception is a disservice to users for the benefit of website owners. It goes against the Mozilla manifesto.

I do not think there is any practical measure we can take here that would have the results you desire.

Mozilla is not on the hook to fix the problem for me personally. This is a community bug report revealing deceptive practice that FF users are exposed to. My fix proposals are also not the only way to fix this defect. They just demonstrate that it’s feasible to fix the bug in a way that gives users information and agency to the degree they prefer.

I got burnt on this again.

Someone trying to get support directed me to visit this snapshot:

https://pasteboard.co/rN1qc5j4AcBU.png

In my Firefox config that page gives “Image not found”. In more normal Firefox configs, that page actually renders an image. I tried fetching it using wget and just got HTML instead of a png image. This is another manifestation of web enshitification that results from Mozilla allowing code to masquerade as something else.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: