429226 - update bulk.cgi/collect.cgi to use new graph server link format

Status: RESOLVED FIXED

(Webtools Graveyard :: Graph Server, defect, P2)

Description

[alice nodelman [:alice] [:anode]]

The new graph server front end uses a different link format; bulk.cgi & collect.cgi are still using the old format.  

Once everyone is using the new format we can switch over the talos machines to use the new front end seamlessly.

Comment 1

[alice nodelman [:alice] [:anode]]

Attachment: change link format for new front end (hg patch) [Checked in]

This changes the links returned by bulk.cgi and collect.cgi to line up with the new front end link format.  This will be an hg only change since it has to do with the new front end.

Once we start using the server code in hg we'll end up automatically switching to the new link format.  

There will be issues on the talos side of the world unless we move the new front end to live in the main directory (ie, graphs.mozilla.org/graphs.html instead of graphs.mozilla.org/g2/graphs.html) - because talos constructs the whole url from the partial sent back from the cgi scripts along with it's knowledge of where the graph server lives.

A simpler solution might be to have the graph server send back complete urls - but then it would have to know it's location.  Is there some way to pull that easily from environment variables?  Could we store it in a local variable that has to be set on graph server installation?  Any advice?

Comment 2

[alice nodelman [:alice] [:anode]]

Attachment: talos patch for greater flexibility in graph server links [Checked in]

When posting data to the graph server talos uses post_file.py.  Right now it only allows for a limited format for links - they must be of the form:

host: www.site.com
selector: /route/to/code/script.cgi

This has worked okay, but it would easier if we could start splitting things out so that this would be okay:

host: www.site.com/route/to/code/
selector: /script.cgi

Once we go with the new front end, server code will live in a separate directory than graph code.  We are going to need this flexibility to have both links to graphs and links to posting data work.

Comment 3

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 326808 [details] [diff] [review]
talos patch for greater flexibility in graph server links [Checked in]

do you even need to do this manual separation of host and selector? It looks like the values should be passed in as arguments, so I'm not sure why this is required.

Also, I'm not sure it's better or not, but an alternative way to split up the host and route/to/selector bits could be to do:

pieces = host.split('/')
if len(pieces) > 1:
    host = pieces[0]
    selector = '/'.join(pieces[1:])

it gets rid of the extra lstrip at least. Tentative r+, but ping me when you get this.

Comment 4

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 326808 [details] [diff] [review]
talos patch for greater flexibility in graph server links [Checked in]

my previous suggestion would require selector to get split('/') as well before rejoining. Probably not worth it.

Comment 5

[alice nodelman [:alice] [:anode]]

talos patch:

Checking in post_file.py;
/cvsroot/mozilla/testing/performance/talos/post_file.py,v  <--  post_file.py
new revision: 1.5; previous revision: 1.4
done

Comment 6

[alice nodelman [:alice] [:anode]]

Comment on attachment 326792 [details] [diff] [review]
change link format for new front end (hg patch) [Checked in]

Checked in to hg in revision 93ece9c5a5ae.

Comment 7

[alice nodelman [:alice] [:anode]]

This is still missing updates to all the talos config files to point at the new location of bulk.cgi - the old front end had it located at graphs.mozilla.org/bulk.cgi but once we switch to the new front end it will live at graphs.mozilla.org/server/bulk.cgi.

We'll only be able to push such a patch once the new front end is located at graphs.mozilla.org instead of graphs.mozilla.org/g2.

Comment 8

[alice nodelman [:alice] [:anode]]

Attachment: switch talos boxes reporting to graphs-stage to new bulk.cgi link [Checked in]

Now that graphs-stage is hg code only we need to update the config files used by talos boxes that report to graphs-stage to use the new location of bulk.cgi.

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 330319 [details] [diff] [review]
switch talos boxes reporting to graphs-stage to new bulk.cgi link [Checked in]

*stamp*

Comment 10

[alice nodelman [:alice] [:anode]]

Comment on attachment 330319 [details] [diff] [review]
switch talos boxes reporting to graphs-stage to new bulk.cgi link [Checked in]

Checking in perf-staging/configs/sample.config;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perf-staging/configs/sample.config,v  <--  sample.config
new revision: 1.6; previous revision: 1.5
done
Checking in tryperfmaster/configs/sample.config;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/tryperfmaster/configs/sample.config,v  <--  sample.config
new revision: 1.6; previous revision: 1.5
done