clobberer fails in 17.0.9 ESR jobs which still use Python 2.5



Release Engineering
5 years ago
3 months ago


(Reporter: nthomas, Assigned: nthomas)


Firefox Tracking Flags

(Not tracked)



(1 attachment)



5 years ago
Calling clobberer: /tools/python/bin/python /builds/slave/rel-m-esr17-firefox_tag/scripts/scripts/release/../..//clobberer/ -s scripts -s buildprops.json release-mozilla-esr17 release-mozilla-esr17-firefox_tag rel-m-esr17-firefox_tag linux-ix-slave02
Checking clobber URL:
Error contacting server
program finished with exit code 1

Going to redo the sendchange to restart, which should avoid any issues with schedulers and rebuilding the tag builder.

Comment 1

5 years ago
This was from When I redid the sendchange, it got linux-ix-slave06 and that also failed. So this looks like doesn't have a complete set of netblocks to whitelist. 

The builder only linux-ix-slave01/02/06 configured, plus half a dozen bld-centos5-32-vmw-001-006.

Comment 2

5 years ago
This is kinda weird. If I make a request from my machine I get a 403 response; since I'm not a build slave this is expected. But the response is cached, so when a build machine makes a request:

cltbld@linux-ix-slave02 ~]$ curl -si ""
HTTP/1.1 403 Forbidden
Server: Apache
Content-Type: text/html; charset=iso-8859-1
Date: Wed, 11 Sep 2013 08:59:49 GMT
Content-Length: 220
Connection: Keep-Alive
X-Cache-Info: cached

Once the cache expires:
HTTP/1.1 200 OK
Server: Apache
Cache-Control: max-age=0
Content-Type: text/html; charset=UTF-8
Date: Wed, 11 Sep 2013 09:05:14 GMT
Expires: Wed, 11 Sep 2013 09:05:14 GMT
X-Powered-By: PHP/5.3.3
X-Cache-Info: not cacheable; response specified max-age <= 0
Content-Length: 50

Note this isn't cached. Any thoughts here Dustin ? Can we make Zeus never cache, or fix the Apache config for LDAP auth so prevents caching ? 

I wouldn't have caused the original 403, so someone maybe being malicious (or just curious) or there's some set of slaves not in a whitelist. My IP begins 121.73.
Flags: needinfo?(dustin)

Comment 3

5 years ago
Scratch that theory. Here's the verbose output:

Checking clobber URL:<snip>
Traceback (most recent call last):
  File "/builds/slave/rel-m-esr17-firefox_tag/scripts/scripts/release/../..//clobberer/", line 191, in <module>
    clobberURL, branch, builder, my_builddir, slave, master)
  File "/builds/slave/rel-m-esr17-firefox_tag/scripts/scripts/release/../..//clobberer/", line 146, in getClobberDates
    data = urllib2.urlopen(url, timeout=30).read().strip()
TypeError: urlopen() got an unexpected keyword argument 'timeout'
Error contacting server

This is finding Python 2.5.1 earliest in the path, and doesn't take a timeout arg until 2.6.
Flags: needinfo?(dustin)


5 years ago
Summary: Failure to reach clobberer in tagging for Firefox 17.0.9 ESR → clobberer fails in some 17.0.9 ESR jobs which use old python

Comment 4

5 years ago
Created attachment 802904 [details] [diff] [review]
[tools] Test for python version

This should probably land in mozharness too, to keep it in sync.
Attachment #802904 - Flags: review?(catlee)


5 years ago
Attachment #802904 - Flags: review?(catlee) → review+

Comment 5

5 years ago

Worked, then we hit bug 915075.
Blocks: 911194
Last Resolved: 5 years ago
Component: Releases → General Automation
QA Contact: bhearsum → catlee
Resolution: --- → FIXED
Summary: clobberer fails in some 17.0.9 ESR jobs which use old python → clobberer fails in 17.0.9 ESR jobs which still use Python 2.5
Sigh, can't believe we're still using Python 2.5 :-(
Least we'll be rid of ESR17 soon..
The caching question is one for webops.  Beyond my Zeus/Apache skills.


5 years ago
Blocks: 915356

Comment 8

5 years ago
(In reply to Nick Thomas [:nthomas] from comment #4)
> This should probably land in mozharness too, to keep it in sync. on default
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.