Closed Bug 476635 Opened 11 years ago Closed 11 years ago

Try server not sending logs to tinderbox

Categories

(Release Engineering :: General, defect, P2)

x86
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: catlee, Assigned: catlee)

Details

Attachments

(1 file)

We're now getting this error on sm-try-master:

2009-02-02 23:50:31-0800 [-] Unhandled Error 
        Traceback (most recent call last): 
          File "/tools/buildbot/lib/python2.5/site-packages/buildbot/process/base.py", line 597, in buildFinished 
            self.build_status.buildFinished() 
          File "/tools/buildbot/lib/python2.5/site-packages/buildbot/status/builder.py", line 1174, in buildFinished 
            w.callback(self) 
          File "/tools/Twisted-8.0.1//lib/python2.5/site-packages/twisted/internet/defer.py", line 242, in callback 
            self._startRunCallbacks(result) 
          File "/tools/Twisted-8.0.1//lib/python2.5/site-packages/twisted/internet/defer.py", line 307, in _startRunCallbacks 
            self._runCallbacks() 
        --- <exception caught here> --- 
          File "/tools/Twisted-8.0.1//lib/python2.5/site-packages/twisted/internet/defer.py", line 323, in _runCallbacks 
            self.result = callback(self.result, *args, **kw) 
          File "/tools/buildbot/lib/python2.5/site-packages/buildbot/status/builder.py", line 1696, in _buildFinished 
            w.buildFinished(name, s, results) 
          File "/tools/buildbot/lib/python2.5/site-packages/buildbot/status/mail.py", line 217, in buildFinished 
            return self.buildMessage(name, build, results) 
          File "/tools/buildbot/lib/python2.5/site-packages/buildbot/status/tinderbox.py", line 217, in buildMessage 
            tinderboxLogs += shortText 
        exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 132107: ordinal not in range(128) 

This looks to be caused from mixing unicode and regular strings.  The arguments that are passed in from sendchange look to be unicode strings (e.g. revision, identifier, who), and so when mixed with the regular string data of the logs, we get this error.

I'm not sure if the unicode is new since the upgrade, or if our logs are outputting something differently.

The net result of this is that builds never appear to finish from tinderbox, even though buildbot thinks they're fine.
Priority: -- → P2
So the cause of this is that the HgPoller pulling data from hg.m.o/try is returning change information as unicode.  Which is fine in itself, but occasionally gcc outputs unicode quote characters (e.g. \u2018) in warnings.

The log files are treated as unencoded strings, so when buildbot is constructing the email to send to tinderbox, it tries to append the unencoded string from the logs with some other text that includes unicode generated from the change information...and then explodes.

I'm not sure at this point what changed recently so that this combination of events is causing us problems.

I have a patch running on sm-staging-try-master that should fix this, just waiting for some builds to go through.
This looks like it's working well on sm-staging-try-master.  We need to test this out elsewhere as well.
Attachment #360414 - Flags: review?(bhearsum)
Attachment #360414 - Flags: review?(bhearsum) → review+
This is now running on the main try server, sm-try-master.

I need to test it out on staging-master as well before committing.
Comment on attachment 360414 [details] [diff] [review]
Make sure we're using 'str' objects for revision and author by encoding unicode into UTF-8

changeset:   191:f9b3862e5fef
Attachment #360414 - Flags: checked‑in+ checked‑in+
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Why isn't this fixed on the status side?
Fixing this on the status side would mean patching up buildbot's TinderboxMailNotifier at the very least, which isn't unicode aware.

Making buildbot unicode-safe is a much larger project.  It seemed better to give buildbot data in a format it could safely deal with.
Are bugs filed for that? At least upstream?

IMHO, this is a short sighted decision and is going to cause more trouble in the long term.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.