Closed Bug 1619816 Opened 4 years ago Closed 4 years ago

Perma [l10n B12] make[1]: *** [package-langpack-ko] Error 1 | make installers-ko failed

Categories

(Firefox Build System :: General, defect, P5)

Product:
Firefox Build System
Component:
General
Type:
defect

Tracking

(firefox75 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla75
Tracking Flags:
Tracking Status
firefox75 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: mshal)

References

(Depends on 1 open bug)

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Bug 1619816 - Add PYTHONIOENCODING to desktop_l10n.py; r?#firefox-build-system-reviewers
47 bytes, text/x-phabricator-request
Details | Review

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=291524065&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/W6K-DUKlQSOl6_XbCpmh5w/runs/0/artifacts/public/logs/live_backing.log

[task 2020-03-03T23:31:49.208Z] 23:31:49 INFO - /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init_py3/bin/python -m mozbuild.action.langpack_manifest --locales ko --min-app-ver 75.0a1 --max-app-ver 75.0a1 --app-name 'Firefox Nightly' --l10n-basedir '/builds/worker/workspace/build/l10n-central' --defines /builds/worker/workspace/build/l10n-central/ko/toolkit/defines.inc /builds/worker/workspace/build/l10n-central/ko/browser/defines.inc --langpack-eid 'langpack-ko@firefox.mozilla.org' --input ../../dist/xpi-stage/locale-ko
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>hee</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>ho073@naver.com</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>Hoony</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>Hyeonseok Shin</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>Jennie Park</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>Jinkyu Yi</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>jsrtyu1120@gmail.com</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>junwookapa@naver.com</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>Kim YoungCheon</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>knr1210@gmail.com</em:contributor>
[task 2020-03-03T23:31:49.483Z] 23:31:49 INFO - <em:contributor>kotelking@gmail.com</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>minwook-shin</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>netaras@gmail.com</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>pillarlee16@gmail.com</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>rlaace423@gmail.com</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>Sang Su Lee</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>SeulGi Choi (Chase)</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>SwooWoong Seol</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>Taegeon Lee</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>Yang Myoung-Gwon</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - <em:contributor>Yang, Jongryul</em:contributor>
[task 2020-03-03T23:31:49.484Z] 23:31:49 ERROR - Traceback (most recent call last):
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - File "/usr/lib/python3.5/runpy.py", line 193, in _run_module_as_main
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - "main", mod_spec)
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
[task 2020-03-03T23:31:49.484Z] 23:31:49 INFO - exec(code, run_globals)
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/langpack_manifest.py", line 489, in <module>
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - main(sys.argv[1:])
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/langpack_manifest.py", line 462, in main
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - defines = parse_defines(args.defines)
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/langpack_manifest.py", line 128, in parse_defines
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - pp.do_include(path)
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/preprocessor.py", line 841, in do_include
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - self.handleLine(l)
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/preprocessor.py", line 601, in handleLine
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - self.write(aLine)
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/preprocessor.py", line 483, in write
[task 2020-03-03T23:31:49.485Z] 23:31:49 INFO - self.out.write(filteredLine)
[task 2020-03-03T23:31:49.486Z] 23:31:49 INFO - UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-19: ordinal not in range(128)
[task 2020-03-03T23:31:49.499Z] 23:31:49 INFO - /builds/worker/workspace/build/src/toolkit/locales/l10n.mk:220: recipe for target 'package-langpack-ko' failed
[task 2020-03-03T23:31:49.499Z] 23:31:49 ERROR - make[1]: *** [package-langpack-ko] Error 1
[task 2020-03-03T23:31:49.499Z] 23:31:49 INFO - make[1]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/browser/locales'
[task 2020-03-03T23:31:49.499Z] 23:31:49 INFO - Makefile:106: recipe for target 'installers-ko' failed
[task 2020-03-03T23:31:49.499Z] 23:31:49 INFO - make: *** [installers-ko] Error 2
[task 2020-03-03T23:31:49.499Z] 23:31:49 ERROR - Return code: 2
[task 2020-03-03T23:31:49.500Z] 23:31:49 ERROR - make installers-ko failed
[task 2020-03-03T23:31:49.500Z] 23:31:49 ERROR - failure: repack_locale(ko)
[task 2020-03-03T23:31:49.500Z] 23:31:49 ERROR - # TBPL FAILURE #
[task 2020-03-03T23:31:49.500Z] 23:31:49 WARNING - setting return code to 2
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - Copying /builds/worker/workspace/build/src/browser/config/mozconfigs/linux64/l10n-mozconfig to /builds/worker/workspace/build/src/.mozconfig
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - Reading from file /builds/worker/workspace/build/src/.mozconfig
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - Contents:
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - ac_add_options --with-branding=browser/branding/nightly
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO -
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - . "$topsrcdir/build/mozconfig.no-compile"
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - ac_add_options --disable-nodejs
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO -
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - export MOZILLA_OFFICIAL=1
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO -
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - # Don't autoclobber l10n, as this can lead to missing binaries and broken builds
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - # Bug 1283438
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO - mk_add_options AUTOCLOBBER=
[task 2020-03-03T23:31:49.500Z] 23:31:49 INFO -
[task 2020-03-03T23:31:49.501Z] 23:31:49 INFO - . "$topsrcdir/build/mozconfig.common.override"
[task 2020-03-03T23:31:49.501Z] 23:31:49 INFO - Running command: ['make', 'installers-lij'] in /builds/worker/workspace/build/src/obj-firefox/browser/locales
[task 2020-03-03T23:31:49.501Z] 23:31:49 INFO - Copy/paste: make installers-lij
[task 2020-03-03T23:31:49.501Z] 23:31:49 INFO - Using env: (same as previous command)
[task 2020-03-03T23:31:49.516Z] 23:31:49 INFO - make[1]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/browser/locales'
[task 2020-03-03T23:31:49.518Z] 23:31:49 INFO - rm -f -rf ../../dist/xpi-stage/locale-lij
[task 2020-03-03T23:31:49.518Z] 23:31:49 INFO - make[1]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/browser/locales'
[task 2020-03-03T23:31:49.522Z] 23:31:49 INFO - make[1]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/browser/locales'
[task 2020-03-03T23:31:49.526Z] 23:31:49 INFO - make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/browser/locales'
[task 2020-03-03T23:31:49.528Z] 23:31:49 INFO - if ! test -d /builds/worker/workspace/build/l10n-central/lij ; then
[task 2020-03-03T23:31:49.528Z] 23:31:49 INFO - echo 'Error: Automation requires l10n repositories to be checked out: /builds/worker/workspace/build/l10n-central/lij' ;
[task 2020-03-03T23:31:49.528Z] 23:31:49 INFO - exit 1 ; \

From IRC:
tom.prince
apavel|sheriffduty: It looks like it is fallout from a combination of https://hg.mozilla.org/mozilla-central/rev/3fa3425acef20d5a554b6b538b750f8d137f7c74 and https://hg.mozilla.org/l10n-central/ko/rev/0e4c6464e9dbae618d7eae4fc1318e26fc5d1ff8

Francesco/Michael please take a look.

Flags: needinfo?(mshal)
Flags: needinfo?(francesco.lodolo)

I'm testing out a fix on try now, will let you know when it finishes.

Flags: needinfo?(mshal)

Pike, didn't you recently add a l10n locale that should have caught this before the merge? Or did the locale you added not have some unicode characters in its manifest (which looks like what broke here)?

Flags: needinfo?(l10n)
Assignee: nobody → mshal
Status: NEW → ASSIGNED

That was bug 1619816 which added 'ar'. I guess no locale had unicode in MOZ_LANGPACK_CONTRIBUTORS prior to https://hg.mozilla.org/l10n-central/ko/rev/0e4c6464e9dbae618d7eae4fc1318e26fc5d1ff8?

Pushed by mshal@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/443bdf34c668
Add PYTHONIOENCODING to desktop_l10n.py; r=glandium

I don't think there's anything to add at this point.

P.S. The bug mshal wanted to reference in comment 5 is bug 1615589

Flags: needinfo?(l10n)
Flags: needinfo?(francesco.lodolo)

https://hg.mozilla.org/mozilla-central/rev/443bdf34c668

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
status-firefox75: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla75
See Also: → 1619937
See Also: → 1619938
Depends on: 1619939

Filed a few bugs, as this bug discovered a few bugs.

The trigger here was that there was junk content with unicode content. The l10n dashboard shows the error, the build removes the error in the merge dir, but the build doesn't read the file from there. Also filed a bug on pontoon to catch these at translation time.

The fix here is good in its own right, though.

The actual MOZ_LANGPACK_CONTRIBUTORS entry was just ascii, fwiw ;-). Ah, the beauty of fuzz testing with l10n.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: