Sunbird Mac l10n builds failing due to requiring license unpack

RESOLVED FIXED

Status

RESOLVED FIXED
9 years ago
7 years ago

People

(Reporter: standard8, Assigned: gozer)

Tracking

Trunk
All
Mac OS X
Dependency tree / graph
Bug Flags:
blocking-calendar1.0 +

Details

Attachments

(2 attachments, 3 obsolete attachments)

Created attachment 394379 [details]
Log

The Sunbird Mac l10n builds are failing since the switch to the new buildbot code which does l10n merge.

I'm not sure what broke this, it could be the recent switch to the new l10n code that also enables l10n merge.

Typical Log:

http://build.mozillamessaging.com/buildbot/production/builders/Sunbird%20MacOSX%2010.4%20comm-1.9.1%20sunbird%20l10n/builds/1962/steps/shell_3/logs/stdio

Comment 1

9 years ago
gah. We just found out today that Tiger's hdiutil is crappy enough to not work without manual intervention, i.e. accepting that license.

This broke on Tiger with bug 498500, and backing that out would break all SeaMonkey L10n repacks, nightlies as well as releases.
Depends on: 498500
Flags: blocking-calendar1.0?
Yeah, we need to fix this for 1.0alpha1 obviously. Stefan, I trust you to + such bugs, go ahead and do so in the future.
Flags: blocking-calendar1.0? → blocking-calendar1.0+
(Assignee)

Updated

9 years ago
Depends on: 510208
Any update on this regression ? L10n builds are now missing since 7 weeks.
(Assignee)

Comment 4

9 years ago
Created attachment 398194 [details] [diff] [review]
bring back unpack's ability to accept EULAs WIP1

I've put this patch together, and with it, I've been able to run 'make unpack' successfully for a Sunbird build.

I am not sure of what the best way for a product to communicate the fact a EULA needs to be accepted. For now, setting UNPACK_ACCEPT_LICENSE in product/locale/Makefile.in does the trick.

Feedback on this approach and testing on Win32 would be appreciated.

Comment 5

9 years ago
The patch sounds wrong to me. This is not about a specific product needing to use the expect approach, it's about Tiger needing it.
(Assignee)

Comment 6

9 years ago
(In reply to comment #5)
> The patch sounds wrong to me. This is not about a specific product needing to
> use the expect approach, it's about Tiger needing it.

Are you sure, all my development/testing so far was done on Leopard and Snow Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?

But in any case, altering when to enable this workaround will be a simple change.

Comment 7

9 years ago
(In reply to comment #6)
> Are you sure, all my development/testing so far was done on Leopard and Snow
> Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?

The Leopard and newer hdiutil doesn't forcibly ask for the license every time and doesn't need the expect script, the Tiger one does. Our problem here is that the Sunbird build machines are running on Tiger and have that even worse version of hdiutil. That tool is the problem, not the DMGs themselves.

> But in any case, altering when to enable this workaround will be a simple
> change.

True, but we should do it the right way so we can remove that hacky workaround where we desupport Tiger.
(Assignee)

Comment 8

9 years ago
(In reply to comment #7)
> (In reply to comment #6)
> > Are you sure, all my development/testing so far was done on Leopard and Snow
> > Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?
> 
> The Leopard and newer hdiutil doesn't forcibly ask for the license every time
> and doesn't need the expect script, the Tiger one does. 

That strange, I tried 'make unpack' on my Leopard box, and it got stuck asking for the license, so it might have something to do with the DMG itself, no ?

> Our problem here is
> that the Sunbird build machines are running on Tiger and have that even worse
> version of hdiutil. That tool is the problem, not the DMGs themselves.

And that shouldn't be too hard a situation to detect

> > But in any case, altering when to enable this workaround will be a simple
> > change.
> 
> True, but we should do it the right way so we can remove that hacky workaround
> where we desupport Tiger.

IMO, the right thing would be conditional on the version of hdutil instead of the version of OS X, no ?

Comment 9

9 years ago
(In reply to comment #8)
Interesting, I never saw it on Leopard boxes, but any I could test on were machines configured for our build reference platform. Maybe it depends on the version of the tools, either the ones used to generate the DMGs or to unpack them - maybe on the Xcode version used? Is hdiutil part of the OS or of Xcode?
(Assignee)

Comment 10

9 years ago
Created attachment 398433 [details] [diff] [review]
bring back unpack's ability to accept EULAs WIP2

A simpler version that simply relies on the OS X version to switch between unpack methods. Thoughts?
Attachment #398194 - Attachment is obsolete: true

Comment 11

9 years ago
Comment on attachment 398433 [details] [diff] [review]
bring back unpack's ability to accept EULAs WIP2

>-hdiutil attach -verbose -noautoopen -mountpoint $MOUNTPOINT "$DMG_PATH"
>+# Thunderbird 2.0.0.x (and others) have a license file and build on
>+# 10.4. For these, we need to use an expect script to mount them.

I think "have a license" is misleading, we probably can just say they build on 10.4 and use an older hdiutil that need a license acknowledgment, or something like that. Your comment sounds like it would be a problem with Thunderbird itself, but it's Apple who did a sucky cmdline tool. :P

Other than that, this looks fine for me.
(Assignee)

Comment 12

9 years ago
Created attachment 398461 [details] [diff] [review]
bring back unpack's ability to accept EULAs on OS X Tiger v1

Here is a complete version that should work for Sunbird. Ready for review.
Attachment #398433 - Attachment is obsolete: true
Is expect available on the mac box? I've seen systems where this isn't installed by default.

Comment 14

9 years ago
(In reply to comment #13)
> Is expect available on the mac box? I've seen systems where this isn't
> installed by default.

It's what we used before switching to the new script that broke you, this is only conditionally restoring the old behavior, so if that box ever worked, it has expect. I'd guess it's part of Xcode or so.
So, who should officially review this? If I were to review it, I'd give r+ codewise, but I can't test this since no mac.
Does this implies that the Sunbird/Lightning build server requires update? For compatibility reasons it probably should use the same build environment as Thunderbird.

Comment 17

9 years ago
A toolkit build system peer or the owner needs to review it.

And yes, the real issue is that the Sunbird build machine is using an older operating system version as all our other 1.9.1-based products. Not sure if the same is true for Lightning, but as it's an extension, it doesn't need to repack DMGs and doesn't run into that issue, this is Sunbird only.
I just wonder if the still expected lifetime of Sunbird warrants getting a newer build machine for it (or adding it to a newer build machine pool).
Setting up build machinery shouldn't be the first hurdle to saving Sunbird, if someone steps up.

Anyway, I think we should upgrade the lightning build machines, this way we can avoid problems that may show up if core decides to depend on 10.6 or a newer version of Xcode.
(Assignee)

Comment 19

9 years ago
(In reply to comment #18)
> Setting up build machinery shouldn't be the first hurdle to saving Sunbird, if
> someone steps up.
> 
> Anyway, I think we should upgrade the lightning build machines, this way we can
> avoid problems that may show up if core decides to depend on 10.6 or a newer
> version of Xcode.

In that case, you'd want to upgrade the build box to 10.5. Just to be clear, Kairo, if these builds were made on 10.5, this patch would be unnecessary ?

Because, if so, upgrading the build box is another perfectly valid option, one that's also closer to what Thunderbird is doing (we build on 10.5 only now)

Comment 20

9 years ago
(In reply to comment #19)
> In that case, you'd want to upgrade the build box to 10.5. Just to be clear,
> Kairo, if these builds were made on 10.5, this patch would be unnecessary ?

Yes, upgrading the build box to 10.5 and the Xcode version we usually use on 10.5 as described in the refplatform docs should solve this problem once an for all as we have a different hdiutil version with different problems there - but for the problems of the new version we have a workaround in place already ;-)

> Because, if so, upgrading the build box is another perfectly valid option, one
> that's also closer to what Thunderbird is doing (we build on 10.5 only now)

Right, and because of that, we're not seeing this problem on Thunderbird (or Firefox or SeaMonkey, for that matter).
(In reply to comment #19)
> Because, if so, upgrading the build box is another perfectly valid option, one
> that's also closer to what Thunderbird is doing (we build on 10.5 only now)

How long would this take (including time until you have time to take care) ?
(Assignee)

Comment 22

9 years ago
(In reply to comment #21)
> (In reply to comment #19)
> > Because, if so, upgrading the build box is another perfectly valid option, one
> > that's also closer to what Thunderbird is doing (we build on 10.5 only now)
> 
> How long would this take (including time until you have time to take care) ?

First, you'd need MoCo IT to reimage that Xserve to the 10.5 refplatform image. Once that's done, should be very quick to have it producing builds again.

Once that's done, it's probably an hour of my time to have buildbot setup on it, so I can do it pretty much anytime, except during the upcoming beta4 build period.

Best order of business is probably filing a MoCo IT bug for the re-imaging, and seeing how soon they could get to it.
(Assignee)

Comment 24

9 years ago
Temporarly disabled l10n repack jobs for Sunbird, they are failing anyways and waste build time.
(Assignee)

Comment 25

9 years ago
cb-xserve03 is back up and running leopard. Only issue is that some of the bits of the reference image (/tools/*) are Intel only binaries, so I'll have to rebuild these.
Summary: Sunbird Mac l10n builds failing due to requiring license unpack → Sunbird: Upgrade cb-xserve03 to OS X Leopard
(Assignee)

Comment 26

9 years ago
cb-xserve03 has the required packages rebuilt. I just restarted buildbot on it. Should start picking up pending builds.
(Assignee)

Comment 27

9 years ago
Back to producing Sunbird/Lightning builds, all GREEN
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
(Assignee)

Comment 28

9 years ago
Created attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

And the l10n repacks jobs fail in exactly the same way, bummer. Turns out that the Sunbird DMG have an extra license resource in the DMG itself, and I really suspect that's the culprit.

One can either use attachment 398461 [details] [diff] [review] to teach make unpack to use expect and accept the DMG license, or one can use this attachment to remove the license in question from the DMG resource. That solution feels simple to me, and nothing except Sunbird seems to be doing that. Changing issues surrounding licenses might be more complex than I can anticipate.
Attachment #398461 - Attachment is obsolete: true
(Assignee)

Updated

9 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 29

9 years ago
Interesting. i totally agree that making Sunbird the same as everybody else is the best solution for this.
Did other used to have this maybe? We don't have the about:rights bar, not sure if this has legal consequences?

Comment 31

9 years ago
SeaMonkey also doesn't have about:rights (yet), and I can't remember hearing anything about anyone else having this or needing it for legal consequences, but it might have been there for someone in some past - not since 1.9.* though or we would have caught it with using the same unpack scripts all over various places (L10n repackaging, release package verification in buildbot, and apparently talos).
Comment on attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

Well, I guess since Linux doesn't have a license displayed on unpack either, we can go ahead and remove it.
Attachment #400217 - Flags: review+
(Assignee)

Comment 33

9 years ago
(In reply to comment #32)
> (From update of attachment 400217 [details] [diff] [review])
> Well, I guess since Linux doesn't have a license displayed on unpack either, we
> can go ahead and remove it.

Then the patch just needs review, not sure who I should request it from. Fallen ?
(Assignee)

Comment 34

9 years ago
Comment on attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

changeset:   3782:78ad69924a87
tag:         tip
user:        Philippe M. Chiasson <gozer@mozillamessaging.com>
date:        Mon Sep 14 10:50:34 2009 -0400
summary:     Bug 510348 - Remove Sunbird license resource from the DMG on Mac OS X. r=Fallen
Attachment #400217 - Attachment description: Remove license resource from the DMG → [checked in] Remove license resource from the DMG
(Assignee)

Comment 35

9 years ago
I forced a nightly this afternoon, and once it completed, l10n repacks fired proprely and succeeded.
Status: REOPENED → RESOLVED
Last Resolved: 9 years ago9 years ago
Resolution: --- → FIXED
Summary: Sunbird: Upgrade cb-xserve03 to OS X Leopard → Sunbird Mac l10n builds failing due to requiring license unpack

Updated

7 years ago
Assignee: nobody → gozer
You need to log in before you can comment on or make changes to this bug.