Closed Bug 619009 Opened 11 years ago Closed 11 years ago

[SeaMonkey] cn-sea-qm-centos5-01 and cn-sea-qm-win2k3-01 fail to post to graph server, since upgrade to buildbot 0.8

Categories

(mozilla.org Graveyard :: Server Operations, task)

task
Not set
major

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: sgautherie, Assigned: cshields)

References

Details

(Keywords: regression, Whiteboard: [perma-purple])

"Intermittent", but very often, on both SM 2.0 and 2.1:
{
...
<h1>Forbidden</h1>
<p>You don't have permission to access /server/collect.cgi
on this server.</p>
...
}

Related to builbot change to v0.8 !?
(Maybe only noticed since then ??)

Maybe some builders have access and not others !?

PS: This is what made me notice bug 619006, before I noticed this actual cause.
This is only semi-dependent on the buildbot 0.8 upgrade, as it's just that it probably uncovered what had been happening for some time.

All boxes hosted in San Jose succeed to post after I resolved bug 618668, but cn-sea-qm-centos5-01, hosted in Amsterdam, still fails and gets a 401 from the graph server.

Alice, any idea why that is?
Depends on: 567035
Summary: Lots of "graph server post results complete failed", since 2010.12.12 → cn-sea-qm-centos5-01 fails to post to graph server
(In reply to comment #1)
> cn-sea-qm-centos5-01, hosted in Amsterdam, still fails and gets a 401 from the
> graph server.

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292326749.1292330723.18571.gz&fulltext=1
WINNT 5.2 comm-central-trunk leak test build on 2010/12/14 03:39:09
s: cn-sea-qm-win2k3-01
too.

*****

Ftr,

Previous builds logged:
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey2.0/1291957989.1291958663.2934.gz&fulltext=1
Linux comm-1.9.1 build on 2010/12/09 21:13:09
s: cn-sea-qm-centos5-01
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey2.0/1291995837.1291999905.8755.gz&fulltext=1
Linux comm-1.9.1 build on 2010/12/10 07:43:57
s: cn-sea-qm-centos5-01
{
======== BuildStep started ========
graph server post ok
=== Output ===
TinderboxPrint: <a title="codesighs" href='http://graphs.mozilla.org/graph.html#tests=[{"test":31,"branch":19,"machine":583}]'>Z:19.6MB</a>
=== Output ended ===
======== BuildStep ended ========
}

New builds log:
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey2.0/1292221103.1292225142.6235.gz&fulltext=1
Linux comm-1.9.1 build on 2010/12/12 22:18:23
s: cn-sea-qm-centos5-01
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey2.0/1292300335.1292300839.10077.gz&fulltext=1
Linux comm-1.9.1 build on 2010/12/13 20:18:55
s: cn-sea-qm-centos5-01
{
======== BuildStep started ========
graph server post results complete failed
=== Output ===
python /builds/slave/comm-1.9.1-linux/tools/buildfarm/utils/retry.py -s 5 -t 120 -r 8 python /builds/slave/comm-1.9.1-linux/tools/buildfarm/utils/graph_server_post.py --server graphs.mozilla.org --selector /server/collect.cgi --branch SeaMonkey2.0 --buildid 20101212221843 --sourcestamp aa7f017b0531 --resultsname Linux_comm-1.9.1 --properties-file properties.json --timestamp 1292221103
...
}

The executed command (and URI) had a major change...
Component: Build Config → Release Engineering
Flags: in-testsuite-
QA Contact: build-config → release
Summary: cn-sea-qm-centos5-01 fails to post to graph server → cn-sea-qm-centos5-01 and cn-sea-qm-win2k3-01 fail to post to graph server, since upgrade to buildbot 0.8
Whiteboard: [perma-purple]
Blocks: 567035
No longer depends on: 567035
I would suggest an IT bug - the graph server does block attempts to send data to it unless the sending machine is on an ip whitelist, your Amsterdam box probably isn't listed.
Flags: in-testsuite-
Product: SeaMonkey → mozilla.org
QA Contact: release → release
Summary: cn-sea-qm-centos5-01 and cn-sea-qm-win2k3-01 fail to post to graph server, since upgrade to buildbot 0.8 → [SeaMonkey] cn-sea-qm-centos5-01 and cn-sea-qm-win2k3-01 fail to post to graph server, since upgrade to buildbot 0.8
Version: Trunk → other
Serge, "IT bug" means Server Ops, not RelEng.
Putting this to the correct component.


cn-sea-qm-centos5-01 and cn-sea-qm-win2k3-01, which run into those problems, are in the .nl colo on the L10n ESX server.

The actual error we get for them is:

Process stderr:
results not added, response: 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /server/collect.cgi
on this server.</p>
<hr>
<address>Apache/2.2.3 (Red Hat) Server at graphs.mozilla.org Port 80</address>
</body></html>
Assignee: nobody → server-ops
Component: Release Engineering → Server Operations
QA Contact: release → mrz
I have added cn-sea-qm-centos5-01 and cn-sea-qm-win2k3-01 to the access list for collect.cgi and bulk.cgi on graphs.mozilla.org

Let me know if the problem persists.
Assignee: server-ops → cshields
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Bug still there:

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292958266.1292963516.10600.gz
Linux comm-central-trunk build on 2010/12/21 11:04:26
s: cn-sea-qm-centos5-01

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292981279.1292991020.9190.gz
WINNT 5.2 comm-central-trunk leak test build on 2010/12/21 17:27:59
s: cn-sea-qm-win2k3-01
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292978498.1292984048.8389.gz
Linux comm-central-trunk leak test build on 2010/12/21 16:41:38
s: cn-sea-qm-centos5-01
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292984265.1292986950.22632.gz
Linux comm-central-trunk leak test build on 2010/12/21 18:17:45
s: cn-sea-qm-centos5-01
Sorry, this was my fault, the change was quickly reverted after I made it.  It is in puppet now so you should be good.
Status: REOPENED → RESOLVED
Closed: 11 years ago11 years ago
Resolution: --- → FIXED
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1293047771.1293049268.6288.gz&fulltext=1
Linux comm-central-trunk build on 2010/12/22 11:56:11
s: cn-sea-qm-centos5-01
{
graph server post results complete
...
program finished with exit code 0
elapsedTime=1.323790
}

V.Fixed
Status: RESOLVED → VERIFIED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.