1048270 - Blobber exception: "UnboundLocalError: local variable 'blob_url' referenced before assignment"

Status: RESOLVED INCOMPLETE

(Release Engineering :: General, defect)

Description

[Ed Morley [:emorley]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=45173843&tree=Mozilla-Central
b2g_emulator_vm mozilla-central opt test mochitest-3 on 2014-08-04 05:00:29 PDT for push e6614d8d85f9

slave: tst-linux64-spot-085

06:22:00     INFO -  (blobuploader) - INFO - Uploading /builds/slave/test/build/blobber_upload_dir/emulator-5554.log ...
06:22:01     INFO -  (blobuploader) - INFO - Using https://blobupload.elasticbeanstalk.com
06:22:01     INFO -  (blobuploader) - INFO - Uploading, attempt #1.
06:22:01     INFO -  (blobuploader) - CRITICAL - Unexpected error in client: Traceback (most recent call last):
06:22:01     INFO -    File "/builds/slave/test/build/venv/bin/blobberc.py", line 165, in upload_file
06:22:01     INFO -      blobhash, compress)
06:22:01     INFO -    File "/builds/slave/test/build/venv/bin/blobberc.py", line 236, in post_file
06:22:01     INFO -      verify=cert.where())
06:22:01     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/requests/api.py", line 88, in post
06:22:01     INFO -      return request('post', url, data=data, **kwargs)
06:22:01     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/requests/api.py", line 44, in request
06:22:01     INFO -      return session.request(method=method, url=url, **kwargs)
06:22:01     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
06:22:01     INFO -      resp = self.send(prep, **send_kwargs)
06:22:01     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
06:22:01     INFO -      r = adapter.send(request, **kwargs)
06:22:01     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 327, in send
06:22:01     INFO -      raise ConnectionError(e)
06:22:01     INFO -  ConnectionError: HTTPSConnectionPool(host='blobupload.elasticbeanstalk.com', port=443): Max retries exceeded with url: /blobs/sha512/b01f4eab7c88fe7789b11649ad6a9df09e4196b02c1211cebcaa33d116d14a685d6c74dc923caa9449806d26294f71f917b6f7fcd6dfd3e97014738f5d874e75 (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
06:22:01     INFO -  (blobuploader) - INFO - Done attempting.
06:22:01     INFO -  Traceback (most recent call last):
06:22:01     INFO -    File "/builds/slave/test/build/venv/bin/blobberc.py", line 273, in <module>
06:22:01     INFO -      main()
06:22:01     INFO -    File "/builds/slave/test/build/venv/bin/blobberc.py", line 265, in main
06:22:01     INFO -      manifest_url_file=manifest_url_file)
06:22:01     INFO -    File "/builds/slave/test/build/venv/bin/blobberc.py", line 94, in upload_dir
06:22:01     INFO -      allowed=allowed_to_send(filename, filetype_whitelist))
06:22:01     INFO -    File "/builds/slave/test/build/venv/bin/blobberc.py", line 185, in upload_file
06:22:01     INFO -      return blob_url
06:22:01     INFO -  UnboundLocalError: local variable 'blob_url' referenced before assignment
06:22:01    ERROR - Return code: 1
06:22:01     INFO - rmtree: /builds/slave/test/build/blobber_manifest.txt
06:22:01     INFO - retry: Calling <built-in function remove> with args: ('/builds/slave/test/build/blobber_manifest.txt',), kwargs: {}, attempt #1
06:22:01     INFO - Copying logs to upload dir...
06:22:01     INFO - mkdir: /builds/slave/test/build/upload/logs
program finished with exit code 0
elapsedTime=4871.136295
========= Finished '/tools/buildbot/bin/python scripts/scripts/b2g_emulator_unittest.py ...' (results: 0, elapsed: 1 hrs, 21 mins, 17 secs) (at 2014-08-04 06:22:07.623896) =========

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

Will look into this as soon as I get near my computer, I'm out of town for two days.

Comment 2

[Jordan Lund (:jlund)]

pinging armen just in case this has something to do with it:
'Bug 986112 - Make blobber uploads discoverable'

particularly the blob_url refs in this patch:
https://bug986112.bugzilla.mozilla.org/attachment.cgi?id=8442904

Comment 3

[Ed Morley [:emorley]]

Yeah this was introduced by https://github.com/MihaiTabara/blobuploader/commit/6df9b4ce7dd297b88d9e67647f19fedf3c3c8a29

Comment 4

[Armen [:armenzg]]

Attachment: blobuploader.diff

It seems that both ret and blob_url are initialized within a try block.
That means if there is an exception (which it happened) is as if they had never been initialized.
https://github.com/MihaiTabara/blobuploader/blob/master/blobberc.py#L163

I haven't tested this. Would you be able to look into this? I can help if you can't.

Comment 5

[Armen [:armenzg]]

Could one of you please review this pull request?
https://github.com/MihaiTabara/blobuploader/pull/16

Thanks!

Comment 6

[Chris AtLee [:catlee]]

lgtm, but I don't have write permissions on that repo to merge it in!

Comment 7

[Mihai Tabara [:mtabara]⌚️GMT]

Merged and pushed 1.2.2 version - https://pypi.python.org/pypi/blobuploader
I'll deploy it on puppet tomorrow as I once again have LDAP credentials :-)

Comment 8

[Armen [:armenzg]]

Thank you Mihai.
We can close this bug after it gets deployed.

Just to be explicit, the error "(Caused by <class 'socket.error'>: [Errno 32] Broken pipe)" should not go away with this fix. I assume this bug was not about it with the title of the bug.

Comment 9

[Mihai Tabara [:mtabara]⌚️GMT]

First things first: added armenzg, catlee and rail as collaborators to https://github.com/MihaiTabara/blobuploader. In the near future this should get somewhere in moz repo base, but until then, it's good to have people to backup different tasks around blobuploader.

Comment 10

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Bump blobuploader version to 1.2.2

Merged https://github.com/MihaiTabara/blobuploader/pull/16 and deployed 1.2.2. version.
Pushed into into our repos http://pypi.pub.build.mozilla.org/pub/
Mozharness change needed, see the one-liner patch.

Comment 11

[Ed Morley [:emorley]]

Mass-closing old bugs I filed that have not had recent activity/no longer affect me.