Traffic to mirrors from bouncer way below average

RESOLVED FIXED

Status

mozilla.org Graveyard
Server Operations
--
blocker
RESOLVED FIXED
8 years ago
3 years ago

People

(Reporter: reed, Assigned: oremj)

Tracking

Details

Attachments

(2 attachments)

(Reporter)

Description

8 years ago
In the last few hours, two different mirror admins have reported to #mirrors that their traffic levels have dropped off dramatically. I checked my mirror's levels, too, and I'm seeing the same thing. I'm still getting some traffic, but it's ridiculously low (only 130.68Mbps at weight 25000). Something is wrong...

Updated

8 years ago
Assignee: server-ops → shyam

Comment 1

8 years ago
This should be back - I was testing something and added it with a mirror weight of 50,000.

Are  you still seeing depressed traffic?

Updated

8 years ago
Assignee: shyam → mrz
(Reporter)

Comment 2

8 years ago
(In reply to comment #1)
> Are  you still seeing depressed traffic?

Yeah, I'm down to 121.96Mbps right now.

Comment 3

8 years ago
What are you expecting to see?  I topped out at just under 2Gbps earlier which matches what I've seen in the past on a Friday past peak during a non-release.
(Reporter)

Comment 4

8 years ago
(In reply to comment #3)
> What are you expecting to see?  I topped out at just under 2Gbps earlier which
> matches what I've seen in the past on a Friday past peak during a non-release.

Generally, 1-1.5Gbps on average.

Comment 5

8 years ago
I'm still seeing a bunch of traffic to this Ankeena box even though it's marked as disabled (and no longer showing up on the sentry page).
(Reporter)

Comment 6

8 years ago
Created attachment 442759 [details]
traffic stats over last 24 hours
Created attachment 442770 [details]
ftp.ankara.edu.tr

It's my network traffic last 24 hours
(Assignee)

Comment 8

8 years ago
I've been watching one webhead for about 8 min now and here are the top 10 mirrors the app is selecting:

956, http://www.mirrorservice.org
1019, http://pv-mirror01.mozilla.org
1109, http://mozilla2.snt.utwente.nl
1147, http://mirror.leaseweb.com
1159, http://mirror.yellowfiber.net
1198, http://mozilla3.snt.utwente.nl
1205, http://ftp.sunet.se
1409, http://hyperion.zih.tu-dresden.de
2255, http://ftp.jaist.ac.jp
2434, http://mozilla.mirror.ac.za
(Reporter)

Comment 10

8 years ago
So, why am I (the top mirror at weight 25000) not in that list?
s/you/voxel/ :)
12:55 < oremj> mrz: in the database voxcdn is not active for 3.6.3

I did note this earlier:

Log entry for  [257] () at 2010-04-30 17:17:29 UTC

Note: a FAILED/404 result on a file which is not included in the mozilla-current
      module is okay if you are only rsyncing mozilla-current.

Checking mirror 3347-mozilla.voxcdn.com ...
http://3347-mozilla.voxcdn.com/pub/mozilla.org sent no response after 5 seconds!  Checking recent history...
**** weight 25000 active 0 for http://3347-mozilla.voxcdn.com/pub/mozilla.org
**** weight 25000 active 1 for http://3347-mozilla.voxcdn.com/pub/mozilla.org
**** weight 25000 active 1 for http://3347-mozilla.voxcdn.com/pub/mozilla.org
**** weight 25000 active 1 for http://3347-mozilla.voxcdn.com/pub/mozilla.org
Pattern OK, leaving weight unchanged.
(Reporter)

Comment 13

8 years ago
(In reply to comment #12)
> 12:55 < oremj> mrz: in the database voxcdn is not active for 3.6.3

Uh, why not? I'm pulling mozilla-releases, and I have 3.6.3. http://3347-mozilla.voxcdn.com/pub/mozilla.org/firefox/releases/3.6.3/

> Log entry for  [257] () at 2010-04-30 17:17:29 UTC

Looks like it's missing some information there?

> Checking mirror 3347-mozilla.voxcdn.com ...
> http://3347-mozilla.voxcdn.com/pub/mozilla.org sent no response after 5
> seconds!

Dunno... Other mirrors have been having sporadic red blocks as well... but seems everything is stable on my end.
(Reporter)

Comment 14

8 years ago
(In reply to comment #11)
> s/you/voxel/ :)

Eh, Voxel just donates the bandwidth and the CDN network. The origin server that serves the CDN its content is my personal server (for which I personally pay).
(Assignee)

Updated

8 years ago
Assignee: mrz → jeremy.orem+bugs
(Assignee)

Comment 15

8 years ago
Found the problem.  It's a bug in sentry.
(Assignee)

Comment 16

8 years ago
Fixed in r66666.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Quick 411 on the bug: If you run a query with the Perl MySQL interface that returns the column "id" from two different tables, the latter will overwrite the former. Makes me want to abandon Perl sooner rather than later.

Fix is here: http://github.com/fwenzel/tuxedo/commit/545dc87 and in r66666.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.