Closed Bug 1405407 Opened 2 years ago Closed 2 years ago

[c-c] SeaMonkey Make installers-% "just work", hard-code build internals or provide sane defaults

Categories

(SeaMonkey :: Build Config, enhancement)

SeaMonkey 2.54 Branch
enhancement
Not set

Tracking

(seamonkey2.54 wontfix, seamonkey2.55 fixed)

RESOLVED FIXED
seamonkey2.55
Tracking Status
seamonkey2.54 --- wontfix
seamonkey2.55 --- fixed

People

(Reporter: frg, Assigned: frg)

References

(Depends on 1 open bug)

Details

User Story

./mozilla/mach installers-de should just work.

Hide l10n-merge, and hard-code the directory in which we do that in the build system.

Check out locales from l10n-central if needed. L10NBASEDIR should have a sound default, ~/.mozbuild/l10n-central.
The automatic check-out of l10n-repos is for Nightly only, to not affect beta builds, which require dedicated revisions. Also, I don't bother to update the l10n repos.

wget-en-US' EN_US_BINARYURL should have a sane default. Which I found for desktop. Not sure if there's one for mobile, and if I can deduce that from config.status.

Create a reliable make variable for when you're in an l10n repack task, and when you're not, and make that independent of configure. Also, have one for langpacks explicitly.


Workflows:

For developers:

  ./mozilla/mach package
  ./mozilla/mach build installers-de

For repacks:

  ./mozilla/mach wget-en-US
  ./mozilla/mach build installers-de

Attachments

(7 files, 12 obsolete files)

7.59 KB, patch
iann_bugzilla
: review+
Details | Diff | Splinter Review
870 bytes, patch
Details | Diff | Splinter Review
10.92 KB, patch
iann_bugzilla
: review+
Details | Diff | Splinter Review
8.81 KB, patch
iann_bugzilla
: review+
Details | Diff | Splinter Review
931 bytes, patch
iann_bugzilla
: review+
Details | Diff | Splinter Review
1.47 KB, patch
ewong
: review+
Details | Diff | Splinter Review
14.00 KB, patch
iann_bugzilla
: review+
Details | Diff | Splinter Review
+++ This bug was initially created as a clone of Bug #1382632 +++

+++ This bug was initially created as a clone of Bug #1370506 +++

Unsurprisingly l10n is broken in SeaMonkey. We need followup changes in suite for bug 1370506.
Attached patch 1405407-wip.patch (obsolete) — Splinter Review
This patch still doesn't like me. Bug 1231349 applies but now also complains about 'devtools-shim'. Running out of time today so just a wip patch. calendar probably also needs a patch. Wonder if/why this currently works.

+++ snip +++

chrome/userChrome-example.css defaults/profile/chrome/userContent-example.css
Warning: Multiple app locales aren't supported: en-GB,el,en-US,ru,de,sk,sv-SE,pl
Warning: Locale doesn't contain distribution/extensions/debugQA@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/inspector@mozilla.org/
Warning: Locale doesn't contain distribution/extensions/{59c81df5-4b7a-477b-912d-4e0fdf64e5f2}/
Error: Locale doesn't have a manifest entry for 'devtools-shim'
Warning: extensions/langpack-de@chatzilla.mozilla.org/chrome/chrome.de.manifest:1: "locale chatzilla de jar:chatzilla.jar!/locale/d
/chatzilla/" is duplicated. Skipping.
Traceback (most recent call last):
  File "d:/seamonkey/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 59, in <module>
    main()
  File "d:/seamonkey/comm-central/mozilla/toolkit/mozapps/installer/l10n-repack.py", line 55, in main
    non_resources=args.non_resource, non_chrome=NON_CHROME)
  File "d:\seamonkey\comm-central\mozilla\python\mozbuild\mozpack\packager\l10n.py", line 269, in repack
    _repack(app_finder, l10n_finder, copier, formatter, non_chrome)
  File "c:\mozilla-build\python\Lib\contextlib.py", line 24, in __exit__
    self.gen.next()
  File "d:\seamonkey\comm-central\mozilla\python\mozbuild\mozpack\errors.py", line 131, in accumulate
    raise AccumulatedErrors()
mozpack.errors.AccumulatedErrors
d:/seamonkey/comm-central/mozilla/toolkit/locales/l10n.mk:125: recipe for target 'repackage-zip' failed
mozmake[2]: *** [repackage-zip] Error 1
mozmake[2]: Leaving directory 'd:/seabuild/release/comm-central-15p/obj-x86_64-pc-mingw32/suite/locales'
Makefile:154: recipe for target 'repackage-win32-installer' failed
mozmake[1]: *** [repackage-win32-installer] Error 2
mozmake[1]: Leaving directory 'd:/seabuild/release/comm-central-15p/obj-x86_64-pc-mingw32/suite/locales'
Makefile:167: recipe for target 'repackage-win32-installer-de' failed
mozmake: *** [repackage-win32-installer-de] Error 2
(In reply to Frank-Rainer Grahl (:frg) from comment #0)
> +++ This bug was initially created as a clone of Bug #1382632 +++
> 
> +++ This bug was initially created as a clone of Bug #1370506 +++
> 
> Unsurprisingly l10n is broken in SeaMonkey. We need followup changes in
> suite for bug 1370506.

I'm not following..  you mean l10n was working before?
> I'm not following..  you mean l10n was working before?

No. That was why I wasn't sruprised :) Bug 1231349 still applies. But nightly l10n is now broken for local builds also.

FRG
Attached patch 1405407-20171008-wip.patch (obsolete) — Splinter Review
Almost working. searchplugins list.txt is still problematic.
Attachment #8914859 - Attachment is obsolete: true
Attached patch 1405407-20171008-wip.patch (obsolete) — Splinter Review
de works but other fail with the erro message below. Magic...

Needed to take parts from:
Bug 525438
Bug 780561
Bug 1111607
Bug 1202168


mozmake: Entering directory 'd:/seabuild/release/comm-central-15/obj-x86_64-pc-mingw32/suite/locales'
mozmake: *** No rule to make target 'd:/seabuild/release/comm-central-15/obj-x86_64-pc-mingw32/dist/seamonkey-2.55a1.en-US.win64.zip
', needed by 'd:/seabuild/release/comm-central-15/obj-x86_64-pc-mingw32/dist/unpacked-installer'.  Stop.
Attached patch 1405407-20171008-wip-part2.patch (obsolete) — Splinter Review
Attachment #8916380 - Attachment is obsolete: true
I need to get the extensions out of the way for further testing.

Part 1 Move debugQA to a new extensions dir to match mozilla/browser/extensions structure. Purely cosmetic.
Assignee: nobody → frgrahl
Status: NEW → ASSIGNED
Attachment #8918663 - Flags: review?(iann_bugzilla)
Part 2 enable or disable extensions via confvars.sh. Building Lightning can also be disabled by passing --disable-calendar

Without the extensions the repack should work (bug 1231349).

If this works as intended (it does locally) the enable-calendar options can be removed from all configs later.

Stay tuned for part 3...
Attachment #8918665 - Flags: review?(iann_bugzilla)
Attached patch 1405407-part3-wip.patch (obsolete) — Splinter Review
part 3 but need to check it some more. A little hacky.
Attachment #8916439 - Attachment is obsolete: true
Attachment #8916440 - Attachment is obsolete: true
Attached patch 1405407-part4-wip.patch (obsolete) — Splinter Review
part 4 Mostly Bug 1202168
disable extension building via confvars.sh.

next trying if I can make Lighting buolding work.
Comment on attachment 8919054 [details] [diff] [review]
1405407-part3-wip.patch

Review of attachment 8919054 [details] [diff] [review]:
-----------------------------------------------------------------

Just some fly-by comments.

I'm surprised about the changes in locales on l10n.toml and shipped-locales, what do they refer to?

::: suite/locales/l10n.ini
@@ +8,5 @@
>  
>  [compare]
>  dirs = suite
> +     mozilla/devtools/client
> +     mozilla/devtools/shim

Neither of these will work.

::: suite/locales/l10n.toml
@@ +61,5 @@
>      path = "{mozilla}/toolkit/locales/l10n.toml"
>  
> +[[paths]]
> +    reference = "{mozilla}/devtools/shim/locales/en-US/**"
> +    l10n = "{l}{mozilla}/devtools/shim/**"

No {mozilla} in the l10n pattern.
> and shipped-locales

Different bug. New ms translation. Use it for testing but need to take it out in the final patch.

> Neither of these will work.

Must confess took these from browser. There it is:

> [compare]
> dirs = browser
>     other-licenses/branding/firefox
>     browser/branding/official
>     devtools/client
>     devtools/shim

In c-c locales is located in:
mozilla/devtools/client
mozilla/devtools/shim

Hmm also not sure how this will work out or can be parameterized after Bug 1366607 if mozilla is the top source dir.

> No {mozilla} in the l10n pattern.

Right thanks.

Thanks for the comments. Trying to get a better understanding of this right now. Not yet there :)
(In reply to Frank-Rainer Grahl (:frg) from comment #13)
> > Neither of these will work.
> 
> Must confess took these from browser. There it is:
> 
> > [compare]
> > dirs = browser
> >     other-licenses/branding/firefox
> >     browser/branding/official
> >     devtools/client
> >     devtools/shim
> 
> In c-c locales is located in:
> mozilla/devtools/client
> mozilla/devtools/shim

The paths need to be the same as in l10n. That means that the .ini files won't be able to handle all of it, but only the TOML files can.

> Hmm also not sure how this will work out or can be parameterized after Bug
> 1366607 if mozilla is the top source dir.

That's a good question. We can set {mozilla} to something on the commandline when using the TOML files, so that should work.

> > No {mozilla} in the l10n pattern.
> 
> Right thanks.
> 
> Thanks for the comments. Trying to get a better understanding of this right
> now. Not yet there :)

np
Version 2. The distribution/extensions path needs to be created during building in case no add-on which uses it is bundled. Packaging will fail otherwise. I took the liberty and lifted a makefile.in from TB in case we want to build ChatZilla and DOMi later from the suite/extensions dir. If not this can be simplified
Attachment #8918665 - Attachment is obsolete: true
Attachment #8918665 - Flags: review?(iann_bugzilla)
Attachment #8919380 - Flags: review?(iann_bugzilla)
Attached patch 1405407-part3-wip.patch (obsolete) — Splinter Review
Could you look this one over for the l10n parts. I was able to build cs de en-GB en-US and the new ms (now removed from here) with this patch and part 4 also applied. As far as I see it merge worked and I saw no errors.
Attachment #8919054 - Attachment is obsolete: true
Attachment #8919382 - Flags: feedback?(l10n)
Attached patch 1405407-part4-wip.patch (obsolete) — Splinter Review
Search plugin packing was broken in the previous patch. This is close to how TB and Fx do it and we should be able to switch to list.json easy later.

Tested with cs, de and ms. ms does not seem to have search plugins defined and used google ddg and yahoo.
Attachment #8919055 - Attachment is obsolete: true
Attached patch 1405407-part5-buildmk.patch (obsolete) — Splinter Review
Noticed that ./mozilla/mach build installers-de doesn't work so another patch on top.
Comment on attachment 8919382 [details] [diff] [review]
1405407-part3-wip.patch

Review of attachment 8919382 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm.
Attachment #8919382 - Flags: feedback?(l10n) → feedback+
Attached patch 1405407-part3-l10ntoml.patch (obsolete) — Splinter Review
Part 3 working for me even with Lighting enabled. 

l10n debugQA, DOMi and irc are still broken because of bug 1231349 but that should be fixed there.
Attachment #8919382 - Attachment is obsolete: true
Attachment #8920832 - Flags: review?(iann_bugzilla)
Attachment #8919387 - Attachment is obsolete: true
Attachment #8920833 - Flags: review?(iann_bugzilla)
just used qrefresh
Attachment #8919394 - Attachment is obsolete: true
Attachment #8920834 - Flags: review?(iann_bugzilla)
spoke with ewong yesterday and we should also remove the calendar-upload. This currently should only be done in the TB make.

calendar-upload:
	$(MAKE) -C $(LIGHTNING_PATH) upload AB_CD=$(AB_CD)
works locally because there is no upload but not sure how it is in automation.
Attachment #8921393 - Flags: review?(ewong)
Comment on attachment 8918663 [details] [diff] [review]
1405407-part1-debugqamove.patch

LGTM r=me
Attachment #8918663 - Flags: review?(iann_bugzilla) → review+
Comment on attachment 8919380 [details] [diff] [review]
1405407-part2-extensionsoptional-V2.patch

LGTM r=me
Attachment #8919380 - Flags: review?(iann_bugzilla) → review+
Attachment #8921393 - Flags: review?(ewong) → review+
Need to rebase part 3 after review because Bug 1411169 already added the os=LikeUnix changes to jar.mn
Pushed by frgrahl@gmx.net:
https://hg.mozilla.org/comm-central/rev/ef9be31bf1c2
part 1 Move debugQA from suite to suite/extensions. r=IanN
https://hg.mozilla.org/comm-central/rev/c0f7fd94ed10
part 2 Allow to enable or disable extension building via confvars. r=IanN
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → ASSIGNED
I removed/ the change "+debugQA.jar" from suite/extensions/debugQA/locales/jar.mn in part 1 before pushing.

This was a typo.
Attached patch 1405407-part3-l10ntoml-V2.patch (obsolete) — Splinter Review
rebased part 3
Attachment #8920832 - Attachment is obsolete: true
Attachment #8920832 - Flags: review?(iann_bugzilla)
Attachment #8922471 - Flags: review?(iann_bugzilla)
Small fix for mozilla as topsourcedir only.
Attachment #8922471 - Attachment is obsolete: true
Attachment #8922471 - Flags: review?(iann_bugzilla)
Attachment #8923782 - Flags: review?(iann_bugzilla)
Comment on attachment 8923782 [details] [diff] [review]
1405407-part3-l10ntoml-V3.patch

Seems to make sense r=me
Attachment #8923782 - Flags: review?(iann_bugzilla) → review+
Comment on attachment 8920833 [details] [diff] [review]
1405407-part4-searchplugins.patch

LGTM r=me
Attachment #8920833 - Flags: review?(iann_bugzilla) → review+
Comment on attachment 8920834 [details] [diff] [review]
1405407-part5-buildmk.patch

r=me
Attachment #8920834 - Flags: review?(iann_bugzilla) → review+
Pushed by frgrahl@gmx.net:
https://hg.mozilla.org/comm-central/rev/b27be9c6a171
part 3 Port Bug 1370506 for l10n building to SeaMonkey.  r=IanN
https://hg.mozilla.org/comm-central/rev/11fd208e7654
part 4 Align SeaMonkey searchplugin processing with Firefox and Thunderbird. r=IanN
https://hg.mozilla.org/comm-central/rev/6f26db434531
part 5 Add l10n-related targets to SeaMonkey top-level build.mk. r=IanN
https://hg.mozilla.org/comm-central/rev/47a7768cc342
part 6 Remove uploading Lightning from SeaMonkey make. r=ewong
Status: ASSIGNED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
Target Milestone: --- → seamonkey2.55
No longer blocks: 2.56BulkMalfunctions
Blocks: 1424283
No longer blocks: 1424283
See Also: → 1424283
You need to log in before you can comment on or make changes to this bug.