366379 - Bouncer2 is still getting a lot of download requests

Status: RESOLVED FIXED

(Webtools :: Bouncer, defect)

Description

[J. Paul Reed [:preed]]

Reportedly, bouncer2 is still getting a lot of hits, despite the fact that no snippets point to bouncer2.

This is a tracking bug to figure out why; we're worried the client may be caching some info, and that's why we're seeing hits to bouncer2.

Cc'ing sspitzer.

Comment 1

[(not reading, please use [e-mail] instead)]

thanks for cc'ing me.

firefox should be points to something like https://aus2.mozilla.org/update/2/Firefox/.....

and that snippet should contain urls like:
http://download.mozilla.org/?product=Firefox....

and should not contain urls like:
http://download2.mozilla.org/?product=Firefox....

(where download.mozilla.org is bouncer1 and download2.mozilla.org is bouncer2)

yet, we are seeing lots of hits on download2.mozilla.org.

do I have that right?

Comment 2

[Michael Morgan [:morgamic]]

That's right except that download2 was advertised for a short period of time during the 2.0.0.1 launch.

So the question is more about whether or not clients could download something pointing to download2 then later retry it?

Comment 3

[Dave Miller [:justdave]]

For reference, from my quick look at the logs when we were originally debugging this...  Referers are logged, but there are no referers, at all.  All of the UserAgents are Firefox/2.0 (but varying OSes and platforms).  This seems to imply that it's definitely clients hitting it.  The question is where they're getting the link from or if they got it from that initial couple hours, why they've cached it so long, and why they're still trying after so long (shouldn't they have got it already by now?)

Comment 4

[(not reading, please use [e-mail] instead)]

morgamic / preed / justdave:

From the logs, can you tell if:

1)  people are hitting download2 for partial or complete updates?
2)  if they are asking for 300KB chunks at a time?
3)  what the client platform is?
4)  how much traffic download2 getting?

here's a scenario, but it doesn't explain "a lot of download requests".

1) a user got the "special" updates.xml during the short period of time when it had direct links to download2.mozilla.org.
2) this user was configured to automatically download the install, so then the client would process that update (active-update.xml on disk) in 300K chunks.  (It used to be 64K, but is now 300K, see bug #352853)

but maybe dan veditz can add some ideas.  he's shown me some odd aus / bouncer graphs before, showing large numbers of weekly attempts to get a point release.

perhaps these two things together are the culprit?

Comment 5

[Dave Miller [:justdave]]

(In reply to comment #4)
> From the logs, can you tell if:
> 
> 1)  people are hitting download2 for partial or complete updates?

both.

> 2)  if they are asking for 300KB chunks at a time?

That doesn't get logged, because we don't care about it, all we're giving them is a mirror referral.  They're supposed to be giving that info to the mirrors.

However, I did some packet tracing of the current traffic going in, and most of the requests to download2 do seem to be coming in with byte range requests in them...

> 3)  what the client platform is?

All of them.  (all flavors of windows, both intel and ppc macs, and Linux)

> 4)  how much traffic download2 getting?

From midnight to midnight PST on Sunday January 7th, there were 63890 requests from 10350 unique IPs.

> here's a scenario, but it doesn't explain "a lot of download requests".
> 
> 1) a user got the "special" updates.xml during the short period of time when it
> had direct links to download2.mozilla.org.
> 2) this user was configured to automatically download the install, so then the
> client would process that update (active-update.xml on disk) in 300K chunks. 
> (It used to be 64K, but is now 300K, see bug #352853)
> 
> but maybe dan veditz can add some ideas.  he's shown me some odd aus / bouncer
> graphs before, showing large numbers of weekly attempts to get a point release.

I had been under the impression that the chunks were supposed to be getting requested from the resulting mirror directly, and that bouncer would never see additional requests from the same client unless it had problems connecting to the mirror again.  Based on the presence of seemingly random byte range requests in the request traffic that assumption was obviously wrong.  If bouncer is getting hit for each chunk request, our download stats are going to be WAY out of whack (like several hundred downloads counted for each user that actually downloaded).

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

active-update.xml & updates.xml store download.m.o URLs rather than the mirror the app is bounced too, so at least on app start there will be a request.

Comment 7

[Dave Miller [:justdave]]

Oh, and on Linux and Windows, the app goes away when you close the last window.  So the app would likely be getting launched frequently.

Comment 8

[Michael Morgan [:morgamic]]

Is this still a problem?