Closed
Bug 657359
Opened 14 years ago
Closed 12 years ago
move pre-buildapi reports off build.mozilla.org
Categories
(Release Engineering :: General, defect, P3)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: dustin, Assigned: dustin)
References
Details
(Keywords: buildapi, Whiteboard: [reporting])
Specifically:
http://build.mozilla.org/builds/pending.html
http://build.mozilla.org/builds/running.html
http://build.mozilla.org/builds/pending/
Are any of these still required? Do they only exist for backward-compatibility with some API? Or are they for backward-compatibility with devs' bookmarks? In the latter case, would a 302 redirect work?
Comment 1•14 years ago
|
||
pending.html and running.html are what tbpl links to in its Infrastructure menu.
Assignee | ||
Comment 2•14 years ago
|
||
Are they just linked-to, or are they parsed?
Comment 3•14 years ago
|
||
Just linked-to, we're not that crazy. We parse http://build.mozilla.org/builds/builds-pending.js and http://build.mozilla.org/builds/builds-running.js. Does that also need to change, or do I just need to change the links?
Assignee | ||
Comment 4•14 years ago
|
||
That will also need to change, but we should be more careful about it. I'm not sure if that's part of buildapi (and thus appropriate for buildapi.pub.build.mozilla.org) or distinct; if the latter, we should probably come up with a new vhost for it.
Maybe catlee can weigh in there, when he's back?
Updated•14 years ago
|
Priority: -- → P3
Whiteboard: [buildapi][reporting]
Comment 5•14 years ago
|
||
The three locations in comment #0 are all generated on cruncher and pulled by an rsync to build.m.o. The same applies to
http://build.mozilla.org/builds/builds-4hr.js.gz
http://build.mozilla.org/builds/builds-2011-05-22.js.gz (and many friends)
http://build.mozilla.org/builds/last-job-per-slave.txt
pending.html is a snapshot of https://build.mozilla.org/buildapi/pending, taken every two minutes (cron:nthomas@cruncher). Similar for running.html, and the json equivalents builds-pending.js and builds-running.js. A redirect might be fine if buildapi can handle the load of tbpl users hitting it.
pending/ is my trending graphs for running and pending jobs (cron:nthomas@cruncher). It hits buildapi/pending and running but is otherwise separate. I'd like to keep it going and move it wherever is appropriate.
builds-4hr.js.gz and builds-YY-MM-DD.js.gz are generated from statusdb by reporter.py (cron:catlee@cruncher) - it lives at http://hg.mozilla.org/users/catlee_mozilla.com/bb_db/. I'm not sure if anyone is downloading those files, we'd have to get access to and check the apache logs on build.m.o.
last-job-per-slave.txt is what it says it is, generated by a python script that queries statusdb (cron:nthomas@cruncher). It could move into buildapi but no idea when I'd get to that.
Assignee | ||
Comment 6•14 years ago
|
||
OK, sounds good. We'll add http://reports.pub.build.mozilla.org to host all of these, with a nice menu page. Bug 657784 updated accordingly.
Comment 7•13 years ago
|
||
Did I correctly follow through the chain and see that there will be a *.pub.build.mozilla.org cert, so that the builds-pending.js and builds-running.js which are currently part of the cause of the mixed-content warnings on https://tbpl.mozilla.org/ will work from https://reports.pub.build.mozilla.org/, without the auth requirement that https://build.mozilla.org/builds/builds-pending.js currently has?
Assignee | ||
Comment 8•13 years ago
|
||
I haven't gotten that far, but I'll check with you when we do get there, and see how to make it happen.
Comment 9•13 years ago
|
||
Well, s/reports/builddata/, but tbpl would still like there to be a valid cert for https://builddata.pub.build.mozilla.org/.
Assignee | ||
Comment 10•12 years ago
|
||
So from a look at the logs, the only paths on builddata.pub.build.mozilla.org being used are /buildjson/, which is proxied to buildapi01. I'm looking at what's still accessed on build.mozilla.org, and will report back.
Assignee | ||
Comment 11•12 years ago
|
||
Here's the set of paths accessed from http://build.mozilla.org/builds:
/builds/buildfaster.csv.gz
/builds/builds-xxxx-xx-xx.js.gz
/builds/builds-4hr.js.gz
/builds/builds-pending.js
/builds/builds-running.js
/builds/last-job-per-slave.txt
/builds/last-job-per-slave.txt
/builds/pending.html
/builds/running.html
/builds/pending/*
all of which but the first are succinctly described in comment 5.
This entire directory is rsync'd from cruncher, so I can just rsync it to two places and mirror the content effectively at a new URL. I propose mirroring /builds at
http://builddata.pub.build.mozilla.org/reports
after which time we'd need to clean up a bunch of links and bookmarks and whatnot to point to the new URL. I can add a 301 for the old URLs at any point where it might help.
Does this sound OK?
philor:
A quick tour around tbpl doesn't reveal access to anything on build.mozilla.org (http or https). So, nothing we're changing here affects tbpl, but your request from 6 months ago is a good one and is pretty easy, so it's now bug 780902. Note that this would have the fringe benefit of serving the above data at
https://secure.pub.build.mozilla.org/builddata/reports
Assignee | ||
Comment 12•12 years ago
|
||
Oh, and lest I forget, reports have some particular requirements from Apache, which I will reproduce faithfully:
<Location /reports>
AllowOverride All
Options +Indexes
AddType text/plain .gz
AddEncoding x-gzip .gz
ExpiresActive on
ExpiresDefault "access plus 1 minute"
</Location>
<Directory /var/www/html/build/builds>
Header set Access-Control-Allow-Origin "*"
</Directory>
Assignee | ||
Comment 13•12 years ago
|
||
(this is in place now on the old, still-active cluster)
Assignee | ||
Comment 14•12 years ago
|
||
FYI, this is now ready to roll on the new cluster, too.
So, over to releng to update links as noted in comment 11.
Updated•12 years ago
|
Whiteboard: [buildapi][reporting] → [reporting]
Assignee | ||
Comment 15•12 years ago
|
||
I'd like to wrap bug 604688 up this quarter, if possible. To that end, I'd like to 301
http://build.mozilla.org/builds -> http://builddata.pub.build.mozilla.org/reports
and similarly (although I don't see it used)
https://build.mozilla.org/builds -> http://secure.pub.build.mozilla.org/builddata/reports
Any objections?
Assignee | ||
Updated•12 years ago
|
Assignee: nobody → dustin
Assignee | ||
Comment 16•12 years ago
|
||
Hearing none, so ordered.
Assignee | ||
Comment 17•12 years ago
|
||
Ah, a few minor cleanups first, that I can take care of.
Assignee | ||
Comment 18•12 years ago
|
||
OK, I don't see anything still hitting builds/ here, so I've added the 301.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
You need to log in
before you can comment on or make changes to this bug.
Description
•