Closed Bug 1231349 (operation_babelfish) Opened 8 years ago Closed 4 years ago

[Tracking Bug] Fix l10n repacks

Categories

(SeaMonkey :: Build Config, defect)

SeaMonkey 2.42 Branch
defect
Not set
blocker

Tracking

(seamonkey2.38 unaffected, seamonkey2.39 unaffected, seamonkey2.40 unaffected, seamonkey2.41 unaffected, seamonkey2.42 wontfix, seamonkey2.43 wontfix, seamonkey2.44 wontfix, seamonkey2.45 wontfix, seamonkey2.46 wontfix, seamonkey2.47 wontfix, seamonkey2.48 wontfix, seamonkey2.49esr+ fixed, seamonkey2.50 wontfix, seamonkey2.51 wontfix, seamonkey2.52 wontfix, seamonkey2.63 wontfix, seamonkey2.53+ fixed, seamonkey2.57esr+ fixed)

RESOLVED FIXED
seamonkey2.49
Tracking Status
seamonkey2.38 --- unaffected
seamonkey2.39 --- unaffected
seamonkey2.40 --- unaffected
seamonkey2.41 --- unaffected
seamonkey2.42 --- wontfix
seamonkey2.43 --- wontfix
seamonkey2.44 --- wontfix
seamonkey2.45 --- wontfix
seamonkey2.46 --- wontfix
seamonkey2.47 --- wontfix
seamonkey2.48 --- wontfix
seamonkey2.49esr + fixed
seamonkey2.50 --- wontfix
seamonkey2.51 --- wontfix
seamonkey2.52 --- wontfix
seamonkey2.63 --- wontfix
seamonkey2.53 + fixed
seamonkey2.57esr + fixed

People

(Reporter: adriank, Assigned: ewong)

References

Details

(Keywords: leave-open, Whiteboard: SM2.53.1)

Attachments

(12 files, 8 obsolete files)

224.56 KB, text/x-log
Details
2.39 KB, patch
Details | Diff | Splinter Review
205.22 KB, text/plain
Details
343.06 KB, application/x-xpinstall
Details
880 bytes, patch
iannbugzilla
: review+
iannbugzilla
: approval-comm-aurora+
iannbugzilla
: approval-comm-beta+
Details | Diff | Splinter Review
6.87 KB, patch
Details | Diff | Splinter Review
1.12 KB, patch
frg
: review+
Details | Diff | Splinter Review
1.91 KB, patch
frg
: review+
Details | Diff | Splinter Review
30.91 KB, patch
iannbugzilla
: feedback+
Details | Diff | Splinter Review
7.65 KB, patch
iannbugzilla
: review+
iannbugzilla
: approval-comm-esr60+
Details | Diff | Splinter Review
15.12 KB, patch
frg
: review+
Details | Diff | Splinter Review
15.07 KB, patch
frg
: review+
Details | Diff | Splinter Review
When trying to make a L10n repack using "make installers-{LOCALES}", this fails with:

11:24:20 /home/mozilla/jenkins/workspace/seamonkey-central_linux64/objdir/_virtualenv/bin/python /home/mozilla/jenkins/workspace/seamonkey-central_linux64/mozilla/toolkit/mozapps/installer/l10n-repack.py /home/mozilla/jenkins/workspace/seamonkey-central_linux64/objdir/dist/l10n-stage/seamonkey ../../dist/xpi-stage/locale-de \
11:24:20 	extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}=../../dist/xpi-stage/lightning-de \
11:24:20 	--non-resource defaults/messenger/mailViews.dat defaults/profile/panels.rdf defaults/profile/mimeTypes.rdf defaults/profile/chrome/userChrome-example.css defaults/profile/chrome/userContent-example.css 
11:24:20 Error: Multiple locales aren't supported
11:24:20 Error: Locale doesn't contain distribution/extensions/debugQA@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
11:24:20 Error: Locale doesn't contain distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}/
11:24:20 Error: Locale doesn't contain extensions/langpack-de@chatzilla.mozilla.org/


This was tested with Linux 32 and 64 bit builds.
Creating just langpacks still works.

A similar bug for Tb from the beginning of the year seems to be this one: bug 1147207.
Attached file repack-de.log
Full log of repacking using the "de" locale on Linux 64bit. Errors from other locales do not differ.
Depends on: 1210791
Is TB Bug 1234935 (Thunderbird beta 43 build3 fails) related?
(In reply to Philip Chee from comment #2)
> Is TB Bug 1234935 (Thunderbird beta 43 build3 fails) related?

I don't think so - but don't know the system good enough to be sure about this.


Nonetheless: now all stages up to beta are affected by this bug... While we still miss the 2.40/2.41 release - after that this bug needs to be fixed, as otherwise no (localized-)release of 2.42 will be possible until then...
Summary: L10n repacks broken on central (2.42) → L10n repacks broken on with SM 2.42 and never
Summary: L10n repacks broken on with SM 2.42 and never → L10n repacks broken on with SM 2.42 and newer
What needs to be added here: Windows 32 bit and 64 bit builds are also affected.

I think this bug here might have been caused by bug 1216371. Also the timing when things've got broken seems correct.

When replacing "error.fatal()" with "error.warn()" for the two errors, the build finishes successfully. But it's just a bad -workaround- hack...
(In reply to Adrian Kalla [:adriank] from comment #4)
> When replacing "error.fatal()" with "error.warn()" for the two errors, the
> build finishes successfully. But it's just a bad -workaround- hack...

It finishes, but it 1) unpacks the addons listed even though it shouldn't touch them at all, and 2) it strips the locale information that is in them. So, it actually ruins the build in the L10n sense. :(

What we need is to have the following files *untouched* by the locale exchanging process and just copied/packed into the repacked package:
distribution/extensions/debugQA@mozilla.org.xpi
distribution/extensions/inspector@mozilla.org.xpi
distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
extensions/langpack-de@chatzilla.mozilla.org.xpi

They exist in l10n-stage before this repack python script run.

Mike, is there anything we can set to achieve that?
Flags: needinfo?(mh+mozilla)
(In reply to Robert Kaiser from comment #5)
> (In reply to Adrian Kalla [:adriank] from comment #4)
> > When replacing "error.fatal()" with "error.warn()" for the two errors, the
> > build finishes successfully. But it's just a bad -workaround- hack...
> 
> It finishes, but it 1) unpacks the addons listed even though it shouldn't
> touch them at all, and 2) it strips the locale information that is in them.
> So, it actually ruins the build in the L10n sense. :(
> 
> What we need is to have the following files *untouched* by the locale
> exchanging process and just copied/packed into the repacked package:
> extensions/langpack-de@chatzilla.mozilla.org.xpi

This one should already be ignored per https://dxr.mozilla.org/mozilla-central/rev/c55bcb7c777ea09431b4d16903ed079ae5632648/toolkit/mozapps/installer/l10n-repack.py#25

> distribution/extensions/debugQA@mozilla.org.xpi
> distribution/extensions/inspector@mozilla.org.xpi
> distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi

I'd argue the problem is that those are not localized in the first place. It's fair that they don't have localized strings (yet), but they should go through locale merge like everything else, and then, like everything else, things will go into place.

BTW, {59...}.xpi is chatzilla, why is the chatzilla locale in a separate addon?
Flags: needinfo?(mh+mozilla)
(In reply to Mike Hommey [:glandium] from comment #6)
> (In reply to Robert Kaiser from comment #5)
> > What we need is to have the following files *untouched* by the locale
> > exchanging process and just copied/packed into the repacked package:
> > extensions/langpack-de@chatzilla.mozilla.org.xpi
> 
> This one should already be ignored per
> https://dxr.mozilla.org/mozilla-central/rev/
> c55bcb7c777ea09431b4d16903ed079ae5632648/toolkit/mozapps/installer/l10n-
> repack.py#25

From all I see, it isn't though, it gets completely stripped from the staging directory before failing with comment #0 messages, at least when I run repavck locally.

> > distribution/extensions/debugQA@mozilla.org.xpi
> > distribution/extensions/inspector@mozilla.org.xpi
> > distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
> 
> I'd argue the problem is that those are not localized in the first place.
> It's fair that they don't have localized strings (yet), but they should go
> through locale merge like everything else, and then, like everything else,
> things will go into place.

Inspector is localized (it contains all localizations existing, just as it does on AMO - and we *need* to ship the same thing as package on AMO because add-on updates will be actually shipped from there) but the process right now strips away all of those local3es from the package and also needlessly unpacks it though it should be a packaged add-on that is ignored by the repack process.

DebugQA is a debug-only add-on that we ship as unlocalizable and never goes into any releases at all (IMHO, we should nowadays just not ship it but it's not my decision any more, I'm just another community member these days, though i still am the de localizer and want to help that L10n works).

> BTW, {59...}.xpi is chatzilla, why is the chatzilla locale in a separate
> addon?

Because an add-on langpack in this form was deemed to be the proper way to add localization to an add-on that is on AMO as an en-US-only add-on. Of course, the decision was made a long time ago, but it's what we have and we should be able to deal with in it SeaMonkey. As it stands right now, the existing code doesn't work for SeaMonkey at all. We need some way to fix things, the easiest way would be if the scripts could just ignore a known set of xpi files in the extensions directories and not unpack or repack those at all. In fact, I think the only xpi-packaged add-on that ever needs repackaging in any form is Lightning.
Blocks: SM2.46
As a test if we add to the list:
> > distribution/extensions/debugQA@mozilla.org.xpi
> > distribution/extensions/inspector@mozilla.org.xpi
> > distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
Does it still error on those?
Flags: needinfo?(ewong)
Flags: needinfo?(akalla)
(In reply to Ian Neal from comment #10)
> As a test if we add to the list:
> > > distribution/extensions/debugQA@mozilla.org.xpi
> > > distribution/extensions/inspector@mozilla.org.xpi
> > > distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
> Does it still error on those?

This is what I get today:

/mnt/mozilla/build/seamonkey/_virtualenv/bin/python /mnt/mozilla/hg/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py /mnt/mozilla/build/seamonkey/dist/l10n-stage/seamonkey ../../dist/xpi-stage/locale-de \
        extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}=../../dist/xpi-stage/lightning-de \
        --non-resource defaults/messenger/mailViews.dat defaults/profile/panels.rdf defaults/profile/mimeTypes.rdf defaults/profile/chrome/userChrome-example.css defaults/profile/chrome/userContent-example.css 
Error: Multiple locales aren't supported
Error: Locale doesn't contain distribution/extensions/debugQA@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Error: Locale doesn't contain distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}/
Error: Locale doesn't contain extensions/langpack-de@chatzilla.mozilla.org/
Traceback (most recent call last):
  File "/mnt/mozilla/hg/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 60, in <module>
    main()
  File "/mnt/mozilla/hg/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 56, in main
    non_resources=args.non_resource, non_chrome=NON_CHROME)
  File "/mnt/mozilla/hg/comm-central/mozilla/python/mozbuild/mozpack/packager/l10n.py", line 250, in repack
    _repack(app_finder, l10n_finder, copier, formatter, non_chrome)
  File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/mnt/mozilla/hg/comm-central/mozilla/python/mozbuild/mozpack/errors.py", line 131, in accumulate
    raise AccumulatedErrors()
mozpack.errors.AccumulatedErrors
(In reply to Ian Neal from comment #10)
> As a test if we add to the list:
> > > distribution/extensions/debugQA@mozilla.org.xpi
> > > distribution/extensions/inspector@mozilla.org.xpi
> > > distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
> Does it still error on those?

I don't know, as for my builds I always apply the hack for this bug since first discovering this bug ( https://bitbucket.org/adrianer/seamonkey-build/src/dcc8d243bf3b1027ecec596deebf76e04f2da78c/bug1231349_workaround.patch?at=default&fileviewer=file-view-default ).
I also guess, that debugQA and if there are xpis or folders depends on the branch, as you'll not find debugQA on beta and release.
Flags: needinfo?(akalla)
(In reply to Ian Neal from comment #10)
> As a test if we add to the list:
> > > distribution/extensions/debugQA@mozilla.org.xpi
> > > distribution/extensions/inspector@mozilla.org.xpi
> > > distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
> Does it still error on those?

Yes.  Currently figuring a way to fix this.
Flags: needinfo?(ewong)
Maybe this helps. Does output more information and sets the locale to the l10n if found in the app dir (highly speculative if this is right)

A stated above the problem seems to be with app.locales and only Chatzilla, DOMi and debugQA.

Picks up more than one language from the app.locales only. If I build de I get en_US from manifests in debugQA and Chatzilla and some others too from DOMi.

If ignoring the warnings doesn't work then the easiest way is probably to change the addons to contain only the same language as the rest of the build.
this is a hack..  it 'unhorks' the repack; but I don't know if it's
right.  Currently running a new build 6 on the loaner.  then I'm going to
try a repack for the CA locale.
Comment on attachment 8799610 [details] [diff] [review]
nifty hack to 'unhork' the repack.

hoping to apply this patch for the repack. (still waiting for win32's build6
to finish).
Comment on attachment 8799610 [details] [diff] [review]
nifty hack to 'unhork' the repack.

was supposed to ask for feedback on that previous post...

Ian, Adrian..  can either or both of you test out this patch with
your locales?

(I'll cancel this feedback when I get my own results... just hope
someone might get it faster..)
Attachment #8799610 - Flags: feedback?(iann_bugzilla)
Attachment #8799610 - Flags: feedback?(akalla)
Comment on attachment 8799610 [details] [diff] [review]
nifty hack to 'unhork' the repack.

There are one or two indentation issues, but as it is a hack...

>+++ b/toolkit/locales/l10n.mk
>@@ -118,7 +118,8 @@ ifdef MOZ_STUB_INSTALLER
> endif
> 	$(PYTHON) $(MOZILLA_DIR)/toolkit/mozapps/installer/l10n-repack.py $(STAGEDIST) $(DIST)/xpi-stage/locale-$(AB_CD) \
> 		$(MOZ_PKG_EXTRAL10N) \
>-		$(if $(filter omni,$(MOZ_PACKAGER_FORMAT)),$(if $(NON_OMNIJAR_FILES),--non-resource $(NON_OMNIJAR_FILES)))
>+		$(if $(filter omni,$(MOZ_PACKAGER_FORMAT)),$(if $(NON_OMNIJAR_FILES),--non-resource $(NON_OMNIJAR_FILES))) \
>+		--ignorelist inspector@ {e2fda1a4-7 {59c81df5-4b lightning chatzilla
I don't think lightning should be in this list.

Not had chance to test though.
Attached patch wip... (obsolete) — Splinter Review
I was hoping to test this out on the windows loaner, but as it stands,
I'm hitting bug 1293943 (on Linux*, it's not busting there..  colour me
very stumped, but it's probably an env issue...just haven't figured 
what it is)... applying that patch, I get bug 1307371.

So if someone can test it out locally... appreciate it.
Attachment #8799610 - Attachment is obsolete: true
Attachment #8799610 - Flags: feedback?(iann_bugzilla)
Attachment #8799610 - Flags: feedback?(akalla)
Attached file repack log
Attached patch hack that unhorks the de repack (obsolete) — Splinter Review
asking for feedback here for:

http://ftp.mozilla.org/pub/seamonkey/candidates/2.46-candidates/build6/unsigned/win32/de/

I'm not entirely sure if this works, tbh.  

1) Can't see ChatZilla
2) Can't see Calendar..
Attachment #8799670 - Attachment is obsolete: true
Flags: needinfo?(frgrahl)
nvm.. it isn't working 

both chatzilla and calendar don't appear in DE but in en-US, it's ok.
Flags: needinfo?(frgrahl)
I uploaded a patch for DebugQA in bug 1238767. Won't help with 2.46 but I am sure a similar patch is needed for DOMi and the language pack for Chatzilla needs to go to fix the first error 'Error: Multiple locales aren't supported' here.
Per Adrians suggestion I started over with a fresh 2.46 local compile and objectdirs for both en-US and de. I only applied his "hack" patch. This time the locale dirs in the extensions were not wiped out so ewong should check again if he finds something fishy.

I did a make installer with the locale and a make installers-de. first one ende up in dist\install\sea and was fine overall. The other ended up in dist\l10n-stage and had the extensions in distribution/extensions not packed but the locales were fine. 

tested: 
mozmake installer
mozmake package

from suite/locales:
mozmake libs
mozmake libs-de 
mozmake installers-de
mozmake repackage-zip
mozmake repackage-zip-de
mozmake langpack
mozmake-langpack-de
(In reply to Adrian Kalla [:adriank] from comment #12)
> (In reply to Ian Neal from comment #10)
> > As a test if we add to the list:
> > > > distribution/extensions/debugQA@mozilla.org.xpi
> > > > distribution/extensions/inspector@mozilla.org.xpi
> > > > distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}.xpi
> > Does it still error on those?
> 
> I don't know, as for my builds I always apply the hack for this bug since
> first discovering this bug (
> https://bitbucket.org/adrianer/seamonkey-build/src/
> dcc8d243bf3b1027ecec596deebf76e04f2da78c/bug1231349_workaround.
> patch?at=default&fileviewer=file-view-default ).
> I also guess, that debugQA and if there are xpis or folders depends on the
> branch, as you'll not find debugQA on beta and release.

As well as the above patch also had to add to suite/locales/Makefile.in (around line 251 after the endif for NIGHTLY_BUILD):
MOZ_PKG_EXTRAL10N += extensions/langpack-$(AB_CD)@chatzilla.mozilla.org=$(DIST)/xpi-stage/chatzilla-$(AB_CD)

Not sure happens when we build for a locale that isn't localised for Chatzilla...
(In reply to Ian Neal from comment #25)
> As well as the above patch also had to add to suite/locales/Makefile.in
> (around line 251 after the endif for NIGHTLY_BUILD):
> MOZ_PKG_EXTRAL10N +=
> extensions/langpack-$(AB_CD)@chatzilla.mozilla.org=$(DIST)/xpi-stage/
> chatzilla-$(AB_CD)
> 
> Not sure happens when we build for a locale that isn't localised for
> Chatzilla...

Sorry, should have said, without the above change the langpack for CZ doesn't seem to get included in extensions/
(In reply to Ian Neal from comment #25)
> As well as the above patch also had to add to suite/locales/Makefile.in
> (around line 251 after the endif for NIGHTLY_BUILD):
> MOZ_PKG_EXTRAL10N +=
> extensions/langpack-$(AB_CD)@chatzilla.mozilla.org=$(DIST)/xpi-stage/
> chatzilla-$(AB_CD)
> 
> Not sure happens when we build for a locale that isn't localised for
> Chatzilla...

I think you've just found the solution for bug 1244467 :)
See Also: → 1311115
The packaging of extensions seem to include directories, which throws the l10n repacks off (this affects Windows packaging).   This needs a m-r patch(forthcoming)
Attachment #8805829 - Flags: review?(iann_bugzilla)
Comment on attachment 8801413 [details] [diff] [review]
hack that unhorks the de repack

>+++ b/python/mozbuild/mozpack/files.py
>@@ -697,7 +697,7 @@ class MinifiedJavaScript(BaseFile):
> 
> class BaseFinder(object):
>     def __init__(self, base, minify=False, minify_js=False,
>-        minify_js_verify_command=None):
>+            minify_js_verify_command=None):
Unneeded?

>+++ b/toolkit/locales/l10n.mk
>@@ -118,7 +118,8 @@ ifdef MOZ_STUB_INSTALLER
> endif
> 	$(PYTHON) $(MOZILLA_DIR)/toolkit/mozapps/installer/l10n-repack.py $(STAGEDIST) $(DIST)/xpi-stage/locale-$(AB_CD) \
> 		$(MOZ_PKG_EXTRAL10N) \
>-		$(if $(filter omni,$(MOZ_PACKAGER_FORMAT)),$(if $(NON_OMNIJAR_FILES),--non-resource $(NON_OMNIJAR_FILES)))
>+		$(if $(filter omni,$(MOZ_PACKAGER_FORMAT)),$(if $(NON_OMNIJAR_FILES),--non-resource $(NON_OMNIJAR_FILES))) \
>+		--ignorelist inspector@ {59c81df5-4b modern chatzilla distribution/extensions/{e2fda1a4-7
Could this be a variable e.g. MOZ_PKG_IGNORE then that can be set in suite/locales/Makefile.in

>diff --git a/toolkit/mozapps/installer/l10n-repack.py b/toolkit/mozapps/installer/l10n-repack.py
>--- a/toolkit/mozapps/installer/l10n-repack.py
>+++ b/toolkit/mozapps/installer/l10n-repack.py
>@@ -24,6 +24,7 @@ NON_CHROME = set([
>     'updater.ini',
>     'extensions/langpack-*@*',
>     'distribution/extensions/langpack-*@*',
>+    'distribution/extensions/{*',
Having this in causes the chatzilla xpi to be deleted by the l10n-repack
>     'chrome/**/searchplugins/*.xml',
> ])
Attachment #8801413 - Flags: feedback+
Assignee: nobody → ewong
Status: NEW → ASSIGNED
Comment on attachment 8805829 [details] [diff] [review]
c-r patch to fix the xpi packaging. [part 1]

r/a=me
Attachment #8805829 - Flags: review?(iann_bugzilla)
Attachment #8805829 - Flags: review+
Attachment #8805829 - Flags: approval-comm-release+
Attachment #8805829 - Flags: approval-comm-beta+
Attachment #8805829 - Flags: approval-comm-aurora+
Comment on attachment 8801413 [details] [diff] [review]
hack that unhorks the de repack

With 'distribution/extensions/{*' removed from NON_CHROME in toolkit/mozapps/installer/l10n-repack.py does this work for you?
Attachment #8801413 - Flags: feedback?(frgrahl)
>> repack.py does this work for you?

Upgraded all my trees to 2.47. Will see if I find some time Tuesday to check it.
(In reply to Ian Neal from comment #32)
> Comment on attachment 8801413 [details] [diff] [review]
> hack that unhorks the de repack
> 
> With 'distribution/extensions/{*' removed from NON_CHROME in
> toolkit/mozapps/installer/l10n-repack.py does this work for you?

Sorry, the other bit that I removed was the ignore on lightning related extensions
Attachment #8801413 - Flags: feedback?(frgrahl)
Attached patch Updated repack patch (obsolete) — Splinter Review
This updated patch has the changes that seem to make repack work but not the change to using a variable.
Attachment #8801413 - Attachment is obsolete: true
Attachment #8812589 - Flags: feedback?(frgrahl)
Comment on attachment 8812589 [details] [diff] [review]
Updated repack patch

This doesn't work on Windows as it's choking later on:

if test -d "c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey/distribution/extensions/inspector@mozilla.org.xpi"; then cd c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey/distribution/extensions/inspector@mozilla.org.xpi/; c:/mozilla-build/info-zip/zip.EXE -Dr9mX ../inspector@mozilla.org.xpi.xpi * -x \*/.mkdir.done; cd ..; rm -rf c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey/distribution/extensions/inspector@mozilla.org.xpi; fi
mozmake[3]: Leaving directory 'c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/suite/app'
rm -f -r c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey/uninstall; c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/_virtualenv/Scripts/python.exe c:/builds/slave/rel-c-r-rpk/build/comm-release/mozilla/config/nsinstall.py -D c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey/uninstall; cp ../installer/windows/l10ngen/helper.exe c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey/uninstall; rm -f c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/setup.exe; cp ../installer/windows/l10ngen/setup.exe c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage; 
c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/_virtualenv/Scripts/python.exe c:/builds/slave/rel-c-r-rpk/build/comm-release/mozilla/toolkit/mozapps/installer/l10n-repack.py c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/dist/l10n-stage/seamonkey ../../dist/xpi-stage/locale-de \
	extensions/langpack-de@chatzilla.mozilla.org=../../dist/xpi-stage/chatzilla-de \
	--non-resource defaults/messenger/mailViews.dat defaults/profile/panels.rdf defaults/profile/mimeTypes.rdf defaults/profile/chrome/userChrome-example.css defaults/profile/chrome/userContent-example.css  \
	--ignorelist inspector@ {59c81df5-4b modern chatzilla
Error: Locale doesn't contain distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/
Error: Locale doesn't contain distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/
Traceback (most recent call last):
  File "c:/builds/slave/rel-c-r-rpk/build/comm-release/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 63, in <module>
    main()
  File "c:/builds/slave/rel-c-r-rpk/build/comm-release/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 59, in main
    ignorelist=args.ignorelist)
  File "c:\builds\slave\rel-c-r-rpk\build\comm-release\mozilla\python\mozbuild\mozpack\packager\l10n.py", line 312, in repack
    _repack(app_finder, l10n_finder, copier, formatter, non_chrome, ignorelist=ignorelist)
  File "c:\mozilla-build\python27\Lib\contextlib.py", line 24, in __exit__
    self.gen.next()
  File "c:\builds\slave\rel-c-r-rpk\build\comm-release\mozilla\python\mozbuild\mozpack\errors.py", line 131, in accumulate
    raise AccumulatedErrors()
mozpack.errors.AccumulatedErrors
c:/builds/slave/rel-c-r-rpk/build/comm-release/mozilla/toolkit/locales/l10n.mk:114: recipe for target 'repackage-zip' failed
mozmake.exe[2]: *** [repackage-zip] Error 1
mozmake.exe[2]: Leaving directory 'c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/suite/locales'
Makefile:142: recipe for target 'repackage-win32-installer' failed
mozmake.exe[1]: *** [repackage-win32-installer] Error 2
mozmake.exe[1]: Leaving directory 'c:/builds/slave/rel-c-r-rpk/build/comm-release/objdir-l10n/suite/locales'
Makefile:155: recipe for target 'repackage-win32-installer-de' failed
pymake\..\..\mozmake.exe: *** [repackage-win32-installer-de] Error 2
Sounds like you haven't got the fix for Bug 1244467 in your c-r
Comment on attachment 8812589 [details] [diff] [review]
Updated repack patch

Sorry took a while longer. With the patch I was able to sucessfully build a 2.46 de on top of the current unmodified release branch. DOMi and modern were ok. I also applied Adrian patches for Chatzilla in bug 485871. The resulting Chatzilla xpi and the cZ language pack where ok.

I encountered a problem. If I disabled the language pack cZ would no longer work. As far as I see it this is a general problem not related to any of the patches. The updated 0.9.93 via amo didn't work either until I enabled the language pack again.
Attachment #8812589 - Flags: feedback?(frgrahl) → feedback+
(In reply to Frank-Rainer Grahl from comment #38)
> Comment on attachment 8812589 [details] [diff] [review]
> Updated repack patch
> 
> Sorry took a while longer. With the patch I was able to sucessfully build a
> 2.46 de on top of the current unmodified release branch. (...)

I assume you did not update the Cz and DOMi changesets in client.py? As they should be updated to something newer, imho.


> I also applied Adrian patches for Chatzilla in bug 485871.

I'd advise to be careful with them, as the regular L10n merge stopped working properly for me after that - and I think that I must have broken it with that patches (though, I'm not sure - it also could have been something else...)
>>
I assume you did not update the Cz and DOMi changesets in client.py? As they should be updated to something newer, imho.
<<

No. I didn't want to mess with the code I case this is what caused the builds to fail. 
This is for ewong and the poers in charge to decide but I think DOMi should be upgraded to current tip. Chatzilla should be upgraded to changeset from Bug 1314457. The next sets add additional l10n variables which you and I stumbled over. If connected it will auto upgrade anyway.

>>
I'd advise to be careful with them, as the regular L10n merge stopped working properly for me after that - and I think that I must have broken it with that patches (though, I'm not sure - it also could have been something else...)
<<

I didn't see any problems with them but I would really like to get rid of the separate language pack.

FRG
(In reply to Frank-Rainer Grahl from comment #40)
> I didn't see any problems with them but I would really like to get rid of
> the separate language pack.

The reason for the separate langpack for Cz is to not run into trouble, when Cz updates itself via AMO. To circumvent this, the Cz offered via AMO would have to provide all the locales Cz is localized in SeaMonkey (instead of just en-US) - like DOMi does. Imho, this is the way to go.
Attached patch 246-l10n.patchSplinter Review
l10n patch on top of relbranch which killed cZ and DOMi only. Worked for me.
(In reply to Adrian Kalla [:adriank] from comment #41)
> The reason for the separate langpack for Cz is to not run into trouble, when
> Cz updates itself via AMO. To circumvent this, the Cz offered via AMO would
> have to provide all the locales Cz is localized in SeaMonkey (instead of
> just en-US) - like DOMi does. Imho, this is the way to go.

ChatZilla developers might have an opinion about this => adding CC
Blocks: 902876
Blocks: SM2.48
No longer blocks: SM2.48
No longer blocks: SM2.46
Attached patch 1231349-debugqa-wip.patch (obsolete) — Splinter Review
make installers-% and merge works debugQA is usable for en-US but for other languages the en-US locale files are not packed into the locale. They are in the correct location in the build dir e.g. xpi-stage/locale-de but don't get packed. Seeing no error message. Maybe someone has an idea. Stumped for now. Some of the changes to the l10n files are not needed because we don't localize debugQA but I tried a few times with different options and left them in. The configuration mostly matches formautofill in mozilla-beta. I was unable to get this working with distribution/extensions at all.
Attachment #8923786 - Flags: feedback?(ewong)
On the positive side: Building calendar works with the patch in Bug 1405407. Got a usable de installer including locales out of it.
debugQA was always supposed to be en-US-only and only included in non-release builds.
> debugQA was always supposed to be en-US-only and only included in non-release builds.

I know but the repack chokes on it currently without a fix. So either the xpi needs to be dropped from non en_US builds or the en_US l10n files need to be merged into the l10n build. It is like formautofill in Fx 57 and below (changed in 58).
(In reply to Frank-Rainer Grahl (:frg) from comment #47)
> > debugQA was always supposed to be en-US-only and only included in non-release builds.
> 
> I know but the repack chokes on it currently without a fix. So either the
> xpi needs to be dropped from non en_US builds or the en_US l10n files need
> to be merged into the l10n build. It is like formautofill in Fx 57 and below
> (changed in 58).

is the choking similar to the ones list in #0?
Comment on attachment 8923786 [details] [diff] [review]
1231349-debugqa-wip.patch

Review of attachment 8923786 [details] [diff] [review]:
-----------------------------------------------------------------

As KaiRo pointed out, we don't distribute debugQA in other locales other than en-US, so don't need to try to hammer debugQA into
locales.

the whole repack-build system is complicated as Reimann Integrals.

::: suite/confvars.sh
@@ +49,5 @@
>  MOZ_IRC=
>  MOZ_DOMINSPECTOR=
>  
>  if [[ $MOZ_APP_VERSION == *a* ]]; then
> +  MOZ_DEBUGQA=1

I'm kinda confused with this change along with the removal
of the Makefile.in file.  If you enable MOZ_DEBUGQA when
the MOZ_APP_VERSION is trunk, I'm feeling the building will choke
because of the missing Makefile.in.
Attachment #8923786 - Flags: feedback?(ewong) → feedback-
(In reply to Edmund Wong (:ewong) from comment #49)
> As KaiRo pointed out, we don't distribute debugQA in other locales other
> than en-US, so don't need to try to hammer debugQA into
> locales.

What KaiRo meant is, that debugQA was always in English in all locales - so it just was not supposed to be localized. Including it just in en-US builds and not in all the other ones makes imho absolutely no sense. And, as KaiRo pointed out: debugQA is for nightlies-only. It was never distributed in releases and betas.
> As KaiRo pointed out, we don't distribute debugQA in other locales other than en-US, 

Do we not distribute it or do we not translate it? There is a difference. As far as I see debugQA is distributed in en-US only in locale builds. I don't want to translate it but for distribution the extension needs to go thru repack and the en-US files need to end up 1:1 in the l10n locale dir of the build. So it needs at least a pusedo merge where the files are just copied. That is what formautofill in Fx did till 57. If we only want to distribute in en-US installers and packs the makefiles and package-manifests need to be changed. If we distribute the en-US as en-US in a locale build we hit this bug and the repack fails.

> I'm kinda confused with this change along with the removal
of the Makefile.in file.

This works now after bug 1405407 not not final. Would sure need a retest.
To be clear, the intent is for debugQA to not be localized and be distributed in en-US form in all pre-release builds (nightly, now-not-existing aurora, and beta channels) but not in any release builds.
Alias: operation_babelfish
Summary: L10n repacks broken on with SM 2.42 and newer → [Tracking Bug] Fix l10n repacks
Depends on: 1421122
Depends on: 1422261
Comment on attachment 8934435 [details] [diff] [review]
[custom] need to re-configure as we had updated the source earlier

lgtm
Attachment #8934435 - Flags: review?(frgrahl) → review+
repack installer step is green.. but it's still choking due to
it not using the path to printconfigsetting.py.
Attachment #8934775 - Flags: review?(frgrahl)
Comment on attachment 8934775 [details] [diff] [review]
[custom] need to specify the right path to printconfigsetting.py

Does this take care of the error 2 with calendar I saw in the log?
Attachment #8934775 - Flags: review?(frgrahl) → review+
fixed previous patch
Attachment #8934775 - Attachment is obsolete: true
Attachment #8934864 - Flags: review+
realized my previous patches were actually overkill.
Attachment #8934864 - Attachment is obsolete: true
Attachment #8934888 - Flags: review?(frgrahl)
Attachment #8934888 - Flags: review?(frgrahl) → review+
Depends on: 1424283
Attached patch 1231349-debugqa.patch (obsolete) — Splinter Review

One down. Two to go.

Tested with 2.53 and 2.57 en-US and de

Brittle stuff. Use xpi_name and it breaks. Use USE_EXTENSION_MANIFEST and it breaks. Tray to use anything fancy and it breaks. Basically only the 56 browser\extensions\pocket configuration with some twists works for debugQA.

No merge. Uses en-US locales for other language as before.

Attachment #8923786 - Attachment is obsolete: true
Attachment #9049812 - Flags: review?(iann_bugzilla)
Attachment #9049812 - Flags: approval-comm-esr60?

Works. This is a bit hacky because it copies the en-US locales from the orginal location and uses the other files directly. I was unable to do this with the locales. The copy step might be improved. The resulting cZ is packed differently but seems to work ok including the irc:// protocol.

On the pro side it does not need a cZ fork but it can be done if desired by just adding the files, adjusting the locations and removing the copy steps.

This drops the language pack.

Tested with esr60.

Attachment #9053149 - Flags: feedback?(iann_bugzilla)

No change just fixing the invalid comment line without bug number.

Attachment #9049812 - Attachment is obsolete: true
Attachment #9049812 - Flags: review?(iann_bugzilla)
Attachment #9049812 - Flags: approval-comm-esr60?
Attachment #9054675 - Flags: review?(iann_bugzilla)
Attachment #9054675 - Flags: approval-comm-esr60?
Comment on attachment 9053149 [details] [diff] [review]
chatZilla repack patch

f=me
seems to work though the chatzilla.manifest file is not created in the xpi
Attachment #9053149 - Flags: feedback?(iann_bugzilla) → feedback+
Comment on attachment 9054675 [details] [diff] [review]
1231349-debugqa.patch

r/a=me
Attachment #9054675 - Flags: review?(iann_bugzilla)
Attachment #9054675 - Flags: review+
Attachment #9054675 - Flags: approval-comm-esr60?
Attachment #9054675 - Flags: approval-comm-esr60+

Pushed by frgrahl@gmx.net:
https://hg.mozilla.org/comm-central/rev/65ca4bd457a6
Fix Debug QA UI extension for l10n builds. r=IanN

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → ASSIGNED
Keywords: leave-open
Comment on attachment 9054675 [details] [diff] [review]
1231349-debugqa.patch

https://hg.mozilla.org/releases/comm-esr60/rev/b29b4d29e3074d026ecca57d30284d18f8bb69c3
Fix Debug QA UI extension for l10n builds r=IanN a=IanN
Regressions: 1559419

Unbitrotted l10n ignorelist patch for 2.49.5 / ESR52 r+ a+

Attachment #8812589 - Attachment is obsolete: true
Attachment #9073415 - Flags: review+
Attachment #9073415 - Flags: approval-comm-esr52+

Version for 2.53 r+ a+

Attachment #9073417 - Flags: review+
Attachment #9073417 - Flags: approval-comm-release+
Target Milestone: seamonkey2.42 → ---

Lets call it fixed with 2.49.

2.53.x repacks work with the workaround and 2.57 local builds too. 2.57 needs further fixes for the cz language pack but this should be done in a follow up bug. Later versions no longer support the extensions which blocked this here.

Target 2.53.1
https://gitlab.com/seamonkey-project/seamonkey-2.53-comm/-/commit/96ef4daec6f166a87abe1d02411d59e1a2024ef2

Status: ASSIGNED → RESOLVED
Closed: 5 years ago4 years ago
Resolution: --- → FIXED
Target Milestone: --- → seamonkey2.49
You need to log in before you can comment on or make changes to this bug.