Closed Bug 1077381 Opened 5 years ago Closed 5 years ago

Android Nightlies failing with "UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)" in generate_browsersearch.py

Categories

(Firefox Build System :: Android Studio and Gradle Integration, defect, blocker)

All
Android
defect
Not set
blocker

Tracking

(Not tracked)

RESOLVED FIXED
mozilla35

People

(Reporter: emorley, Assigned: nalexander)

References

Details

Attachments

(1 file)

eg:
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=443446&repo=mozilla-central

06:57:44 INFO - /builds/slave/m-cen-and-ntly-000000000000000/build/obj-firefox/_virtualenv/bin/python -m mozbuild.action.generate_browsersearch --verbose --srcdir=/builds/slave/m-cen-and-ntly-000000000000000/l10n-central/zh-CN/mobile/chrome --srcdir=/builds/slave/m-cen-and-ntly-000000000000000/build/mobile/locales/en-US/chrome ../res/raw-zh-rCN/browsersearch.json
06:57:44 INFO - Read 3 engines: [u'\u767e\u5ea6', u'Google', u'Google']
06:57:44 ERROR - Traceback (most recent call last):
06:57:44 INFO - File "/tools/python27/lib/python2.7/runpy.py", line 162, in _run_module_as_main
06:57:44 INFO - "__main__", fname, loader, pkg_name)
06:57:44 INFO - File "/tools/python27/lib/python2.7/runpy.py", line 72, in _run_code
06:57:44 INFO - exec code in run_globals
06:57:44 INFO - File "/builds/slave/m-cen-and-ntly-000000000000000/build/python/mozbuild/mozbuild/action/generate_browsersearch.py", line 98, in <module>
06:57:44 INFO - sys.exit(main(sys.argv[1:]))
06:57:44 INFO - File "/builds/slave/m-cen-and-ntly-000000000000000/build/python/mozbuild/mozbuild/action/generate_browsersearch.py", line 76, in main
06:57:44 INFO - print("Default engine is '{default}'.".format(default=default))
06:57:44 INFO - UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)
06:57:44 ERROR - make[2]: *** [../res/raw-zh-rCN/browsersearch.json] Error 1
06:57:44 INFO - make[2]: Leaving directory `/builds/slave/m-cen-and-ntly-000000000000000/build/obj-firefox/mobile/android/base/locales'
06:57:44 ERROR - make[1]: *** [chrome-zh-CN] Error 2
06:57:44 INFO - make[1]: Leaving directory `/builds/slave/m-cen-and-ntly-000000000000000/build/obj-firefox/mobile/android/base/locales'
06:57:44 INFO - make: *** [chrome-zh-CN] Error 2
06:57:44 ERROR - Return code: 2
06:57:44 ERROR - Failed to add locale zh-CN!

That file was added by bug 1065306.

There don't appear to have been any recent commits to:
https://hg.mozilla.org/l10n-central/zh-CN/
yes, zh-CN uses a chinese default search plugin, with a chinese description, thus unicode.

print needs to utf-8 encode
We need codecs for the print function and unicode_literals to let the
format() string stay undecorated.
Attachment #8499708 - Flags: review?(gps)
Duplicate of this bug: 1077853
Assignee: nobody → nalexander
Status: NEW → ASSIGNED
I think we can avoid this problem by toggling --verbose off.  I'm going to land that work-around and trigger Nightlies; we can follow up with real utf-8 careful output.
Keywords: leave-open
Component: Search Activity → Build Config & IDE Support
Hardware: ARM → All
OK, the work-around above gave the expected green Nightly:

http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2014/10/2014-10-05-03-02-05-mozilla-central-android/

We have red N1, N3, and N4 builds.  Investigating.
> We have red N1, N3, and N4 builds.  Investigating.

These all look like Balrog (update service) connectivity and/or database version errors.  Retriggering to see if they're transient.
(In reply to Nick Alexander :nalexander from comment #10)
> > We have red N1, N3, and N4 builds.  Investigating.
> 
> These all look like Balrog (update service) connectivity and/or database
> version errors.  Retriggering to see if they're transient.

Looks like the retriggers are all green \o/
(In reply to Nick Alexander :nalexander from comment #11)
> (In reply to Nick Alexander :nalexander from comment #10)
> > > We have red N1, N3, and N4 builds.  Investigating.
> > 
> > These all look like Balrog (update service) connectivity and/or database
> > version errors.  Retriggering to see if they're transient.
> 
> Looks like the retriggers are all green \o/

Looks like these were updates racing, which is something we should be retrying for. I filed bug 1078248.
Comment on attachment 8499708 [details] [diff] [review]
Encode generate_browsersearch verbose output as utf-8. r=gps

Review of attachment 8499708 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/test/action/test_generate_browsersearch.py
@@ +1,1 @@
> +# -*- coding: utf-8 -*-

Need license header. Creative commons something.
Attachment #8499708 - Flags: review?(gps) → review+
Keywords: leave-open
There's some strange things going on with N1 builds (see [1]), but I see green N1's for e4cfacb76830 [2].  So this looks good.

[1] https://tbpl.mozilla.org/php/getParsedLog.php?id=49814368&tree=Mozilla-Central

[2] https://tbpl.mozilla.org/php/getParsedLog.php?id=49791686&tree=Mozilla-Central)
Flags: qe-verify-
Product: Firefox for Android → Firefox Build System
Target Milestone: Firefox 35 → mozilla35
You need to log in before you can comment on or make changes to this bug.