1405407 - [c-c] SeaMonkey Make installers-% "just work", hard-code build internals or provide sane defaults

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, enhancement)

Description

[Frank-Rainer Grahl (:frg)]

+++ This bug was initially created as a clone of Bug #1382632 +++

+++ This bug was initially created as a clone of Bug #1370506 +++

Unsurprisingly l10n is broken in SeaMonkey. We need followup changes in suite for bug 1370506.

Comment 1

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-wip.patch

This patch still doesn't like me. Bug 1231349 applies but now also complains about 'devtools-shim'. Running out of time today so just a wip patch. calendar probably also needs a patch. Wonder if/why this currently works.

+++ snip +++

chrome/userChrome-example.css defaults/profile/chrome/userContent-example.css
Warning: Multiple app locales aren't supported: en-GB,el,en-US,ru,de,sk,sv-SE,pl
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/[e-mail]/
Warning: Locale doesn't contain distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}/
Error: Locale doesn't have a manifest entry for 'devtools-shim'
Warning: extensions/[e-mail]/chrome/chrome.de.manifest:1: "locale chatzilla de jar:chatzilla.jar!/locale/d
/chatzilla/" is duplicated. Skipping.
Traceback (most recent call last):
  File "d:/seamonkey/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 59, in <module>
    main()
  File "d:/seamonkey/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 55, in main
    non_resources=args.non_resource, non_chrome=NON_CHROME)
  File "d:\seamonkey\comm-central\mozilla\python\mozbuild\mozpack\packager\l10n.py", line 269, in repack
    _repack(app_finder, l10n_finder, copier, formatter, non_chrome)
  File "c:\mozilla-build\python\Lib\contextlib.py", line 24, in __exit__
    self.gen.next()
  File "d:\seamonkey\comm-central\mozilla\python\mozbuild\mozpack\errors.py", line 131, in accumulate
    raise AccumulatedErrors()
mozpack.errors.AccumulatedErrors
d:/seamonkey/comm-central/mozilla/toolkit/locales/l10n.mk:125: recipe for target 'repackage-zip' failed
mozmake[2]: *** [repackage-zip] Error 1
mozmake[2]: Leaving directory 'd:/seabuild/release/comm-central-15p/obj-x86_64-pc-mingw32/suite/locales'
Makefile:154: recipe for target 'repackage-win32-installer' failed
mozmake[1]: *** [repackage-win32-installer] Error 2
mozmake[1]: Leaving directory 'd:/seabuild/release/comm-central-15p/obj-x86_64-pc-mingw32/suite/locales'
Makefile:167: recipe for target 'repackage-win32-installer-de' failed
mozmake: *** [repackage-win32-installer-de] Error 2

Comment 2

[Edmund Wong (:ewong)]

(In reply to Frank-Rainer Grahl (:frg) from comment #0)
> +++ This bug was initially created as a clone of Bug #1382632 +++
> 
> +++ This bug was initially created as a clone of Bug #1370506 +++
> 
> Unsurprisingly l10n is broken in SeaMonkey. We need followup changes in
> suite for bug 1370506.

I'm not following..  you mean l10n was working before?

Comment 3

[Frank-Rainer Grahl (:frg)]

> I'm not following..  you mean l10n was working before?

No. That was why I wasn't sruprised :) Bug 1231349 still applies. But nightly l10n is now broken for local builds also.

FRG

Comment 4

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-20171008-wip.patch

Almost working. searchplugins list.txt is still problematic.

Comment 5

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-20171008-wip.patch

de works but other fail with the erro message below. Magic...

Needed to take parts from:
Bug 525438
Bug 780561
Bug 1111607
Bug 1202168


mozmake: Entering directory 'd:/seabuild/release/comm-central-15/obj-x86_64-pc-mingw32/suite/locales'
mozmake: *** No rule to make target 'd:/seabuild/release/comm-central-15/obj-x86_64-pc-mingw32/dist/seamonkey-2.55a1.en-US.win64.zip
', needed by 'd:/seabuild/release/comm-central-15/obj-x86_64-pc-mingw32/dist/unpacked-installer'.  Stop.

Comment 6

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-20171008-wip-part2.patch

Comment 7

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part1-debugqamove.patch

I need to get the extensions out of the way for further testing.

Part 1 Move debugQA to a new extensions dir to match mozilla/browser/extensions structure. Purely cosmetic.

Comment 8

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part2-extensionsoptional.patch

Part 2 enable or disable extensions via confvars.sh. Building Lightning can also be disabled by passing --disable-calendar

Without the extensions the repack should work (bug 1231349).

If this works as intended (it does locally) the enable-calendar options can be removed from all configs later.

Stay tuned for part 3...

Comment 9

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part3-wip.patch

part 3 but need to check it some more. A little hacky.

Comment 10

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part4-wip.patch

part 4 Mostly Bug 1202168

Comment 11

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-partx-disable-extensions.patch

disable extension building via confvars.sh.

next trying if I can make Lighting buolding work.

Comment 12

[Axel Hecht [:Pike]]

Comment on attachment 8919054 [details] [diff] [review]
1405407-part3-wip.patch

Review of attachment 8919054 [details] [diff] [review]:
-----------------------------------------------------------------

Just some fly-by comments.

I'm surprised about the changes in locales on l10n.toml and shipped-locales, what do they refer to?

::: suite/locales/l10n.ini
@@ +8,5 @@
>  
>  [compare]
>  dirs = suite
> +     mozilla/devtools/client
> +     mozilla/devtools/shim

Neither of these will work.

::: suite/locales/l10n.toml
@@ +61,5 @@
>      path = "{mozilla}/toolkit/locales/l10n.toml"
>  
> +[[paths]]
> +    reference = "{mozilla}/devtools/shim/locales/en-US/**"
> +    l10n = "{l}{mozilla}/devtools/shim/**"

No {mozilla} in the l10n pattern.

Comment 13

[Frank-Rainer Grahl (:frg)]

> and shipped-locales

Different bug. New ms translation. Use it for testing but need to take it out in the final patch.

> Neither of these will work.

Must confess took these from browser. There it is:

> [compare]
> dirs = browser
>     other-licenses/branding/firefox
>     browser/branding/official
>     devtools/client
>     devtools/shim

In c-c locales is located in:
mozilla/devtools/client
mozilla/devtools/shim

Hmm also not sure how this will work out or can be parameterized after Bug 1366607 if mozilla is the top source dir.

> No {mozilla} in the l10n pattern.

Right thanks.

Thanks for the comments. Trying to get a better understanding of this right now. Not yet there :)

Comment 14

[Axel Hecht [:Pike]]

(In reply to Frank-Rainer Grahl (:frg) from comment #13)
> > Neither of these will work.
> 
> Must confess took these from browser. There it is:
> 
> > [compare]
> > dirs = browser
> >     other-licenses/branding/firefox
> >     browser/branding/official
> >     devtools/client
> >     devtools/shim
> 
> In c-c locales is located in:
> mozilla/devtools/client
> mozilla/devtools/shim

The paths need to be the same as in l10n. That means that the .ini files won't be able to handle all of it, but only the TOML files can.

> Hmm also not sure how this will work out or can be parameterized after Bug
> 1366607 if mozilla is the top source dir.

That's a good question. We can set {mozilla} to something on the commandline when using the TOML files, so that should work.

> > No {mozilla} in the l10n pattern.
> 
> Right thanks.
> 
> Thanks for the comments. Trying to get a better understanding of this right
> now. Not yet there :)

np

Comment 15

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part2-extensionsoptional-V2.patch

Version 2. The distribution/extensions path needs to be created during building in case no add-on which uses it is bundled. Packaging will fail otherwise. I took the liberty and lifted a makefile.in from TB in case we want to build ChatZilla and DOMi later from the suite/extensions dir. If not this can be simplified

Comment 16

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part3-wip.patch

Could you look this one over for the l10n parts. I was able to build cs de en-GB en-US and the new ms (now removed from here) with this patch and part 4 also applied. As far as I see it merge worked and I saw no errors.

Comment 17

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part4-wip.patch

Search plugin packing was broken in the previous patch. This is close to how TB and Fx do it and we should be able to switch to list.json easy later.

Tested with cs, de and ms. ms does not seem to have search plugins defined and used google ddg and yahoo.

Comment 18

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part5-buildmk.patch

Noticed that ./mozilla/mach build installers-de doesn't work so another patch on top.

Comment 19

[Axel Hecht [:Pike]]

Comment on attachment 8919382 [details] [diff] [review]
1405407-part3-wip.patch

Review of attachment 8919382 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm.

Comment 20

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part3-l10ntoml.patch

Part 3 working for me even with Lighting enabled. 

l10n debugQA, DOMi and irc are still broken because of bug 1231349 but that should be fixed there.

Comment 21

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part4-searchplugins.patch

Comment 22

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part5-buildmk.patch

just used qrefresh

Comment 23

[Frank-Rainer Grahl (:frg)]

spoke with ewong yesterday and we should also remove the calendar-upload. This currently should only be done in the TB make.

calendar-upload:
	$(MAKE) -C $(LIGHTNING_PATH) upload AB_CD=$(AB_CD)

Comment 24

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part6-caluploadremove.patch

works locally because there is no upload but not sure how it is in automation.

Comment 25

[Ian Neal]

Comment on attachment 8918663 [details] [diff] [review]
1405407-part1-debugqamove.patch

LGTM r=me

Comment 26

[Ian Neal]

Comment on attachment 8919380 [details] [diff] [review]
1405407-part2-extensionsoptional-V2.patch

LGTM r=me

Comment 27

[Frank-Rainer Grahl (:frg)]

Need to rebase part 3 after review because Bug 1411169 already added the os=LikeUnix changes to jar.mn

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/ef9be31bf1c2
part 1 Move debugQA from suite to suite/extensions. r=IanN
https://hg.mozilla.org/comm-central/rev/c0f7fd94ed10
part 2 Allow to enable or disable extension building via confvars. r=IanN

Comment 29

[Frank-Rainer Grahl (:frg)]

I removed/ the change "+debugQA.jar" from suite/extensions/debugQA/locales/jar.mn in part 1 before pushing.

This was a typo.

Comment 30

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part3-l10ntoml-V2.patch

rebased part 3

Comment 31

[Frank-Rainer Grahl (:frg)]

Attachment: 1405407-part3-l10ntoml-V3.patch

Small fix for mozilla as topsourcedir only.

Comment 32

[Ian Neal]

Comment on attachment 8923782 [details] [diff] [review]
1405407-part3-l10ntoml-V3.patch

Seems to make sense r=me

Comment 33

[Ian Neal]

Comment on attachment 8920833 [details] [diff] [review]
1405407-part4-searchplugins.patch

LGTM r=me

Comment 34

[Ian Neal]

Comment on attachment 8920834 [details] [diff] [review]
1405407-part5-buildmk.patch

r=me

Comment 35

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/b27be9c6a171
part 3 Port Bug 1370506 for l10n building to SeaMonkey.  r=IanN
https://hg.mozilla.org/comm-central/rev/11fd208e7654
part 4 Align SeaMonkey searchplugin processing with Firefox and Thunderbird. r=IanN
https://hg.mozilla.org/comm-central/rev/6f26db434531
part 5 Add l10n-related targets to SeaMonkey top-level build.mk. r=IanN
https://hg.mozilla.org/comm-central/rev/47a7768cc342
part 6 Remove uploading Lightning from SeaMonkey make. r=ewong