281682 - RSS Reader is DDoSing sites

Status: RESOLVED WORKSFORME

(Firefox :: Bookmarks & History, defect)

Description

[The Founder]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; Q312461; .NET CLR 1.1.4322; Alexa Toolbar)
Build Identifier: All Versions

I love Firefox... and I want everyone to keep using it.. and if you don't have 
it... please install it... but I have to say one thing.. 

Firefox is a website crasher 

The built in tool with Firefox, the RSS reader... crashes websites.. not 
because Firefox does a bad job at RSS reading, because it does too good of a 
job. 

For the past few weeks I thought I was under a Denial of Service attack.. I 
couldn't understand why 380 or more people per second were hitting Sqlspace... 

It took me a few days to figure it out. Firefox has the built in RSS reader 
that hits that RSS feed like crazy.. We have a total of 3 RSS feeds.. each with 
a default of 50 items... we had for example 380 people online at one time. A 
total of 57,000 items-posts were sent out per second. 

I know this is rehashed from prior.. but it's more than noticed before. 

They come from hundreds of IP addresses... all though the night.. though the 
day.. 

The Active Bookmark that firefox provides has been crashing websites for the 
past few months.. and it's being not though no fault of firefox.. but because 
people never expected that Active Bookmark to actually do that much damage.. 

I have reduced the figure from 50 posts to 15 ... this has reduced the load on 
the server for now, but the point is beware on offering RSS feeds.... not 
because they are bad.. but because the popular firefox browser is growing so 
fast that it's almost a Denial of service attack... it's not like 1 ip that you 
can ban.. you would have to ban 1/2 the internet.... or half your viewers..


Reproducible: Always

Actual Results:  
it crashes servers

Expected Results:  
crashes servers

There needs to be a serious delay in how often the RSS reader updates... every 
1/2 hour for example.

Comment 1

[Dave Miller [:justdave]]

I don't think it even hits that often.  I just checked the logs on my blog,
which has an rdf feed, and I'm only getting hit once per hour per IP with
Firefox useragents.  Maybe your site is just popular?

Are you getting hit by the same IP address with a Firefox useragent more than
once per hour?

Comment 2

[Dave Miller [:justdave]]

No response to followup questions asked of the reporter, and I still can't
reproduce it.

If you can reproduce this, or you're the reporter and have the answers to the
questions that were asked, feel free to reopen it.

Comment 3

[Pascal Chevrel:pascalc (PTO until August 27)]

Reopening. The Spanish Mozilla Portal (mozillaes.org) is experiencing the same
problem, they thought they were experincing DoS attacks but it seems that in
fact all es-ES version of Firefox (which include an RSS feed to the latest
mozilla news in Spanish from this portal) are loading the rss feed like crazy,
killing the server capacities to the point that the portal is about to close.

After shutting down the RSS feed because it was consuming too much of their
bandwidth, the server is now having problems to cope with all the failed hits to
the feed. Unless there are *far* more people using es-ES than downloads numbers
and browser statistics suggest, the RSS feed is loaded dozens of times an hour
per user (multiplicate it with let's say 1 million es-ES firefox and you see the
problem). My guess is that when the RSS feed is not available, firefox
repeatedly tries to access it until it can get a valid feed.

If somebody knows where I can find documentation on how often feeds are loaded
by firefox I am interested

Comment 4

[Pascal Chevrel:pascalc (PTO until August 27)]

marking New.

Comment 5

[Dave Townsend [:mossop]]

See bug 261076 comment 3 for details on how fast firefox can refresh the feed.

Comment 6

[Pascal Chevrel:pascalc (PTO until August 27)]

Dave, thanks for the link but unfortunately i don't understand the calculation.
On what http response header is the calculation made ? expires: age: max-age:?
What if the server sends a no-cache response (which I believe is the case, it's
powered by mambo and mambo always sends it by default) or doesn't send anything
about expiration and cache values? bug 261076 comment 5 seems to suggest that
the current refresh time in this case is every minute, which is frightening and
would explain why their portal is living it as a DoS attack...

Comment 7

[Dave Townsend [:mossop]]

I've taken a look at the code and here is what happens. When firefox grabs the
feed it decides how long to wait to grab it again. If a certain pref is set then
it will take this value (bumping it up to a minute if it is any less), otherwise
it will use an hour. It then checks to see how long till the feed expires from
firefox's cache, if thats a longer time then it uses that instead.

That pref is not set by default in firefox and is not listed in about:config so
you have to find help on how to set it.

So if the feed is set as uncacheable for whatever reason, then chances are
firefox should still only be reloading it every hour.

Hopefully that explains it for you.

Comment 8

[Pascal Chevrel:pascalc (PTO until August 27)]

ok, so since the pref is not set on current 1.0.x versions, the default value
should be 1 hour. And if there is no longer a RSS feed at the url but a blank
page, it doesn't change anything ?

Comment 9

[Dave Townsend [:mossop]]

Forgot to look at that. Urgh, its not great. It looks like if the feed load
fails for any reason firefox will try again in 5 minutes unless I'm reading this
wrong.

Comment 10

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #9)
> Forgot to look at that. Urgh, its not great. It looks like if the feed load
> fails for any reason firefox will try again in 5 minutes unless I'm reading this
> wrong.

The fallback time in case of failure is 1 hour, as far as I can see.
http://lxr.mozilla.org/mozilla/source/browser/components/bookmarks/src/nsBookmarksFeedHandler.cpp#277

Comment 11

[Dave Townsend [:mossop]]

I was going by the failure here which seems to retry every 5 minutes. Not sure what can trigger that though, maybe its network errors rather than invalid feeds?

http://lxr.mozilla.org/mozilla/source/browser/components/bookmarks/src/nsBookmarksFeedHandler.cpp#214

Comment 12

[Mike Connor [:mconnor]]

sorry for bugspam, long-overdue mass reassign of ancient QA contact bugs, filter on "beltznerLovesGoats" to get rid of this mass change

Comment 13

[Marco Bonardo [:mak]]

actually refreshed every hour, after 10 minutes in case of error, this should be WFM.