Closed Bug 1146035 Opened 9 years ago Closed 9 years ago

having a non-ASCII character in the tip changeset's commit message causes builds to fail

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

Product:
Release Engineering
Component:
Applications: MozharnessCore
Platform:
x86_64
All
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dbaron, Unassigned)

References

Details

Attachments

(1 file)

bug1146035
974 bytes, patch
jlund
: review+
mozilla
: checked-in+
Details | Diff | Splinter Review 
Having a non-ASCII character in the tip commit causes builds to fail in automation.  I suspect this is a recent regression, and it's quite serious.

13:10:58 INFO - command: START
13:10:58 INFO - command: hg parent --template {node|short}
13:10:58 INFO - command: cwd: /builds/slave/m-in-lx-0000000000000000000000/build/src
13:10:58 INFO - command: env: {'HGPLAIN': '1'}
13:10:58 INFO - command: END (0.50s elapsed)
13:10:58 INFO - command: output:
13:10:58 INFO - 2cf8496afd5d
13:10:58 INFO - Got revision 2cf8496afd5d
13:10:58 INFO - Return code: 0
13:10:58 INFO - Setting buildbot property comments to Bug 1138157 - Change ScriptedDirectProxyHandler to inherit from BaseProxyHandler. r=efaust
13:10:58 INFO -
13:10:58 INFO - This avoids the fallback behavior from DirectProxyHandler, which can't handle revoked proxies correctly.
13:10:58 INFO -
13:10:58 INFO - Test created by André Bargull and me.
13:10:58 INFO - Writing buildbot properties ['comments'] to /builds/slave/m-in-lx-0000000000000000000000/properties/comments
13:10:58 INFO - Writing to file /builds/slave/m-in-lx-0000000000000000000000/properties/comments
13:10:58 INFO - Contents:
13:10:58 INFO - comments:Bug 1138157 - Change ScriptedDirectProxyHandler to inherit from BaseProxyHandler. r=efaust
13:10:58 INFO -
13:10:58 INFO - This avoids the fallback behavior from DirectProxyHandler, which can't handle revoked proxies correctly.
13:10:58 INFO -
13:10:58 INFO - Test created by André Bargull and me.
13:10:58 FATAL - Uncaught exception: Traceback (most recent call last):
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/base/script.py", line 1285, in run
13:10:58 FATAL - self.run_action(action)
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/base/script.py", line 1226, in run_action
13:10:58 FATAL - self._possibly_run_method("preflight_%s" % method_name)
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/base/script.py", line 1168, in _possibly_run_method
13:10:58 FATAL - return getattr(self, method_name)()
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/mozilla/building/buildbase.py", line 1406, in preflight_build
13:10:58 FATAL - self._checkout_source()
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/mozilla/building/buildbase.py", line 1045, in _checkout_source
13:10:58 FATAL - write_to_file=True)
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/mozilla/buildbot.py", line 100, in set_buildbot_property
13:10:58 FATAL - return self.dump_buildbot_properties(prop_list=[prop_name], file_name=prop_name)
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/mozilla/buildbot.py", line 140, in dump_buildbot_properties
13:10:58 FATAL - return self.write_to_file(file_name, contents)
13:10:58 FATAL - File "/tools/checkouts/mozharness/mozharness/base/script.py", line 437, in write_to_file
13:10:58 FATAL - fh.write(contents)
13:10:58 FATAL - UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 227: ordinal not in range(128)
13:10:58 FATAL - Running post_fatal callback...
13:10:58 ERROR - setting return code to 2 because fatal was called
13:10:58 FATAL - Exiting -1
13:10:58 INFO - Running post-run listener: _summarize
13:10:58 ERROR - # TBPL FAILURE #
Summary: having a non-ASCII character in the tip commit causes builds to fail → having a non-ASCII character in the tip changeset's commit message causes builds to fail
Bug 782312 tracks using unicode everywhere in mozharness, and contains a workaround patch that may help solve this specific symptom.
philor says this is a duplicate of bug 1106342
Attached patch bug1146035 — — Splinter Review 
I think this will help resolve this bug, but I'd love another test pass if possible.

I don't like the nested try/except blocks; if that bothers you, maybe something like

    try:
        fh = open(file_path, open_mode)
    except IOError:
        self.log("%s can't be opened for writing!" % file_path,
                 level=error_level)
        return None  # ?
    try:
        fh.write(contents)
    except UnicodeEncodeError:
        fh.write(contents.encode('utf-8', 'replace'))
    fh.close()
    return file_path

?
Attachment #8581194 - Flags: review?(jlund)
Comment on attachment 8581194 [details] [diff] [review]
bug1146035

Review of attachment 8581194 [details] [diff] [review]:
-----------------------------------------------------------------

nested doesn't really bother me if it's only one deep. as for testing, mh is pretty self contained now thanks to pinning in tree.

Once landed, I'll pin it on a branch and pass a commit with unicode on it.
Attachment #8581194 - Flags: review?(jlund) → review+
I'm guessing this is resolved; please comment and reopen if we still hit this problem.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
actually comment 23 just means this is on our releng production env but not necessarily any c-i build/test jobs as we pin the mh rev to each branch. I have updated the mozharness rev on cedar to point to your commit and triggered a build with a unicode char in the commit message:

https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=32e5083241f5

when this build ^ completes, we should have a good idea if we can resolve this.
Status: RESOLVED → UNCONFIRMED
Ever confirmed: false
Resolution: FIXED → ---
looks good:

11:41:10     INFO - Setting buildbot property comments to testing latest mozharness prod rev - × <- unicode char CLOSED TREE r=me
11:41:10     INFO - Writing buildbot properties ['comments'] to /builds/slave/ced-l64-d-00000000000000000000/properties/comments
11:41:10     INFO - Writing to file /builds/slave/ced-l64-d-00000000000000000000/properties/comments
11:41:10     INFO - Contents:
11:41:10     INFO -  comments:testing latest mozharness prod rev - × <- unicode char CLOSED TREE r=me

let's land this in prod branches
this was picked up with mshal's mh json bump
Status: UNCONFIRMED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: