712674 - Win7 xperf job are running green but they are having urllib2.HTTPError: HTTP Error 504: Gateway Time-out exception

Status: RESOLVED DUPLICATE of bug 779571

(Testing :: Talos, defect)

Description

[Armen [:armenzg]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=8086180&tree=Firefox&full=1

xperf: warning: applying restriction of access for trace processing
in readfile: test.etl.csv.part, firefox
Traceback (most recent call last):
  File "xtalos\etlparser.py", line 305, in ?
    main()
  File "xtalos\etlparser.py", line 299, in main
    alog.post()
  File "xtalos\etlparser.py", line 111, in post
    self.testGroup.submit() 
  File "c:\talos-slave\talos-data\talos\xtalos\mozautolog.py", line 570, in submit
    response_stream = urllib2.urlopen(req)
  File "C:\Python24\lib\urllib2.py", line 130, in urlopen
    return _opener.open(url, data)
  File "C:\Python24\lib\urllib2.py", line 364, in open
    response = meth(req, response)
  File "C:\Python24\lib\urllib2.py", line 471, in http_response
    response = self.parent.error(
  File "C:\Python24\lib\urllib2.py", line 402, in error
    return self._call_chain(*args)
  File "C:\Python24\lib\urllib2.py", line 337, in _call_chain
    result = func(*args)
  File "C:\Python24\lib\urllib2.py", line 480, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 504: Gateway Time-out

Comment 1

[cmtalbert]

These are working again.  I think this happened when the Elastic Search database went offline last week.  https://tbpl.mozilla.org/php/getParsedLog.php?id=8167358&tree=Firefox&full=1

Is there another bug that tracks making a change so that these are not reporting as green if they encounter an issue uploading data?  Or should that patch be attached here?

Comment 2

[Armen [:armenzg]]

I think we could use this bug to fix it.
What happens when it goes offline?
I believe this only affects xperf jobs which are only enable on m-c and some other branch (I think try).
We probably want to ask whoever owns xperf if they still care about this job or not (I don't see it posting to graphs or anywhere else). Not sure if it should show up on tbpl if no one looks for this suite.

Comment 3

[cmtalbert]

These report to autolog right now, because it wasn't possible to change the graphserver enough to deal with their data when we brought them online (and we didn't know what kinds of data would be useful to keep and analyze). So it made more sense to bring them up using a NOSQL database for storage so we could easily keep everything and then pull together the metrics we needed for analysis.  

I think there are people looking at them, and that there have been more requests for more different kinds of data from the xperf system that talos is running.  So, I think they are useful, but we need to really look into this.

I think this should also be considered as we change the graphserver for SfN because ideally we'd be able to provide access to all our talos data in one place, i.e. graphserver.

Net: I think this is a bug that needs to be solved i.e. the builds need to turn red when these tests fail to report.

Comment 4

[Jeff Hammel]

I'm guessing this is still an issue?

Comment 5

[Jeff Hammel]

Since we no longer post to autolog, this is done