510348 - Sunbird Mac l10n builds failing due to requiring license unpack

Status: RESOLVED FIXED

(Calendar :: Build Config, defect)

Description

[Mark Banner (:standard8)]

Attachment: Log

The Sunbird Mac l10n builds are failing since the switch to the new buildbot code which does l10n merge.

I'm not sure what broke this, it could be the recent switch to the new l10n code that also enables l10n merge.

Typical Log:

http://build.mozillamessaging.com/buildbot/production/builders/Sunbird%20MacOSX%2010.4%20comm-1.9.1%20sunbird%20l10n/builds/1962/steps/shell_3/logs/stdio

Comment 1

[Robert Kaiser]

gah. We just found out today that Tiger's hdiutil is crappy enough to not work without manual intervention, i.e. accepting that license.

This broke on Tiger with bug 498500, and backing that out would break all SeaMonkey L10n repacks, nightlies as well as releases.

Comment 2

[Philipp Kewisch [:Fallen][📱📆]]

Yeah, we need to fix this for 1.0alpha1 obviously. Stefan, I trust you to + such bugs, go ahead and do so in the future.

Comment 3

[Stefan Sitter [:ssitter]]

Any update on this regression ? L10n builds are now missing since 7 weeks.

Comment 4

[Philippe M. Chiasson (:gozer)]

Attachment: bring back unpack's ability to accept EULAs WIP1

I've put this patch together, and with it, I've been able to run 'make unpack' successfully for a Sunbird build.

I am not sure of what the best way for a product to communicate the fact a EULA needs to be accepted. For now, setting UNPACK_ACCEPT_LICENSE in product/locale/Makefile.in does the trick.

Feedback on this approach and testing on Win32 would be appreciated.

Comment 5

[Robert Kaiser]

The patch sounds wrong to me. This is not about a specific product needing to use the expect approach, it's about Tiger needing it.

Comment 6

[Philippe M. Chiasson (:gozer)]

(In reply to comment #5)
> The patch sounds wrong to me. This is not about a specific product needing to
> use the expect approach, it's about Tiger needing it.

Are you sure, all my development/testing so far was done on Leopard and Snow Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?

But in any case, altering when to enable this workaround will be a simple change.

Comment 7

[Robert Kaiser]

(In reply to comment #6)
> Are you sure, all my development/testing so far was done on Leopard and Snow
> Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?

The Leopard and newer hdiutil doesn't forcibly ask for the license every time and doesn't need the expect script, the Tiger one does. Our problem here is that the Sunbird build machines are running on Tiger and have that even worse version of hdiutil. That tool is the problem, not the DMGs themselves.

> But in any case, altering when to enable this workaround will be a simple
> change.

True, but we should do it the right way so we can remove that hacky workaround where we desupport Tiger.

Comment 8

[Philippe M. Chiasson (:gozer)]

(In reply to comment #7)
> (In reply to comment #6)
> > Are you sure, all my development/testing so far was done on Leopard and Snow
> > Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?
> 
> The Leopard and newer hdiutil doesn't forcibly ask for the license every time
> and doesn't need the expect script, the Tiger one does. 

That strange, I tried 'make unpack' on my Leopard box, and it got stuck asking for the license, so it might have something to do with the DMG itself, no ?

> Our problem here is
> that the Sunbird build machines are running on Tiger and have that even worse
> version of hdiutil. That tool is the problem, not the DMGs themselves.

And that shouldn't be too hard a situation to detect

> > But in any case, altering when to enable this workaround will be a simple
> > change.
> 
> True, but we should do it the right way so we can remove that hacky workaround
> where we desupport Tiger.

IMO, the right thing would be conditional on the version of hdutil instead of the version of OS X, no ?

Comment 9

[Robert Kaiser]

(In reply to comment #8)
Interesting, I never saw it on Leopard boxes, but any I could test on were machines configured for our build reference platform. Maybe it depends on the version of the tools, either the ones used to generate the DMGs or to unpack them - maybe on the Xcode version used? Is hdiutil part of the OS or of Xcode?

Comment 10

[Philippe M. Chiasson (:gozer)]

Attachment: bring back unpack's ability to accept EULAs WIP2

A simpler version that simply relies on the OS X version to switch between unpack methods. Thoughts?

Comment 11

[Robert Kaiser]

Comment on attachment 398433 [details] [diff] [review]
bring back unpack's ability to accept EULAs WIP2

>-hdiutil attach -verbose -noautoopen -mountpoint $MOUNTPOINT "$DMG_PATH"
>+# Thunderbird 2.0.0.x (and others) have a license file and build on
>+# 10.4. For these, we need to use an expect script to mount them.

I think "have a license" is misleading, we probably can just say they build on 10.4 and use an older hdiutil that need a license acknowledgment, or something like that. Your comment sounds like it would be a problem with Thunderbird itself, but it's Apple who did a sucky cmdline tool. :P

Other than that, this looks fine for me.

Comment 12

[Philippe M. Chiasson (:gozer)]

Attachment: bring back unpack's ability to accept EULAs on OS X Tiger v1

Here is a complete version that should work for Sunbird. Ready for review.

Comment 13

[Philipp Kewisch [:Fallen][📱📆]]

Is expect available on the mac box? I've seen systems where this isn't installed by default.

Comment 14

[Robert Kaiser]

(In reply to comment #13)
> Is expect available on the mac box? I've seen systems where this isn't
> installed by default.

It's what we used before switching to the new script that broke you, this is only conditionally restoring the old behavior, so if that box ever worked, it has expect. I'd guess it's part of Xcode or so.

Comment 15

[Philipp Kewisch [:Fallen][📱📆]]

So, who should officially review this? If I were to review it, I'd give r+ codewise, but I can't test this since no mac.

Comment 16

[Stefan Sitter [:ssitter]]

Does this implies that the Sunbird/Lightning build server requires update? For compatibility reasons it probably should use the same build environment as Thunderbird.

Comment 17

[Robert Kaiser]

A toolkit build system peer or the owner needs to review it.

And yes, the real issue is that the Sunbird build machine is using an older operating system version as all our other 1.9.1-based products. Not sure if the same is true for Lightning, but as it's an extension, it doesn't need to repack DMGs and doesn't run into that issue, this is Sunbird only.
I just wonder if the still expected lifetime of Sunbird warrants getting a newer build machine for it (or adding it to a newer build machine pool).

Comment 18

[Philipp Kewisch [:Fallen][📱📆]]

Setting up build machinery shouldn't be the first hurdle to saving Sunbird, if someone steps up.

Anyway, I think we should upgrade the lightning build machines, this way we can avoid problems that may show up if core decides to depend on 10.6 or a newer version of Xcode.

Comment 19

[Philippe M. Chiasson (:gozer)]

(In reply to comment #18)
> Setting up build machinery shouldn't be the first hurdle to saving Sunbird, if
> someone steps up.
> 
> Anyway, I think we should upgrade the lightning build machines, this way we can
> avoid problems that may show up if core decides to depend on 10.6 or a newer
> version of Xcode.

In that case, you'd want to upgrade the build box to 10.5. Just to be clear, Kairo, if these builds were made on 10.5, this patch would be unnecessary ?

Because, if so, upgrading the build box is another perfectly valid option, one that's also closer to what Thunderbird is doing (we build on 10.5 only now)

Comment 20

[Robert Kaiser]

(In reply to comment #19)
> In that case, you'd want to upgrade the build box to 10.5. Just to be clear,
> Kairo, if these builds were made on 10.5, this patch would be unnecessary ?

Yes, upgrading the build box to 10.5 and the Xcode version we usually use on 10.5 as described in the refplatform docs should solve this problem once an for all as we have a different hdiutil version with different problems there - but for the problems of the new version we have a workaround in place already ;-)

> Because, if so, upgrading the build box is another perfectly valid option, one
> that's also closer to what Thunderbird is doing (we build on 10.5 only now)

Right, and because of that, we're not seeing this problem on Thunderbird (or Firefox or SeaMonkey, for that matter).

Comment 21

[Philipp Kewisch [:Fallen][📱📆]]

(In reply to comment #19)
> Because, if so, upgrading the build box is another perfectly valid option, one
> that's also closer to what Thunderbird is doing (we build on 10.5 only now)

How long would this take (including time until you have time to take care) ?

Comment 22

[Philippe M. Chiasson (:gozer)]

(In reply to comment #21)
> (In reply to comment #19)
> > Because, if so, upgrading the build box is another perfectly valid option, one
> > that's also closer to what Thunderbird is doing (we build on 10.5 only now)
> 
> How long would this take (including time until you have time to take care) ?

First, you'd need MoCo IT to reimage that Xserve to the 10.5 refplatform image. Once that's done, should be very quick to have it producing builds again.

Once that's done, it's probably an hour of my time to have buildbot setup on it, so I can do it pretty much anytime, except during the upcoming beta4 build period.

Best order of business is probably filing a MoCo IT bug for the re-imaging, and seeing how soon they could get to it.

Comment 23

[Philipp Kewisch [:Fallen][📱📆]]

Filed bug 515128 to re-image

Comment 24

[Philippe M. Chiasson (:gozer)]

Temporarly disabled l10n repack jobs for Sunbird, they are failing anyways and waste build time.

Comment 25

[Philippe M. Chiasson (:gozer)]

cb-xserve03 is back up and running leopard. Only issue is that some of the bits of the reference image (/tools/*) are Intel only binaries, so I'll have to rebuild these.

Comment 26

[Philippe M. Chiasson (:gozer)]

cb-xserve03 has the required packages rebuilt. I just restarted buildbot on it. Should start picking up pending builds.

Comment 27

[Philippe M. Chiasson (:gozer)]

Back to producing Sunbird/Lightning builds, all GREEN

Comment 28

[Philippe M. Chiasson (:gozer)]

Attachment: [checked in] Remove license resource from the DMG

And the l10n repacks jobs fail in exactly the same way, bummer. Turns out that the Sunbird DMG have an extra license resource in the DMG itself, and I really suspect that's the culprit.

One can either use attachment 398461 [details] [diff] [review] to teach make unpack to use expect and accept the DMG license, or one can use this attachment to remove the license in question from the DMG resource. That solution feels simple to me, and nothing except Sunbird seems to be doing that. Changing issues surrounding licenses might be more complex than I can anticipate.

Comment 29

[Robert Kaiser]

Interesting. i totally agree that making Sunbird the same as everybody else is the best solution for this.

Comment 30

[Philipp Kewisch [:Fallen][📱📆]]

Did other used to have this maybe? We don't have the about:rights bar, not sure if this has legal consequences?

Comment 31

[Robert Kaiser]

SeaMonkey also doesn't have about:rights (yet), and I can't remember hearing anything about anyone else having this or needing it for legal consequences, but it might have been there for someone in some past - not since 1.9.* though or we would have caught it with using the same unpack scripts all over various places (L10n repackaging, release package verification in buildbot, and apparently talos).

Comment 32

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

Well, I guess since Linux doesn't have a license displayed on unpack either, we can go ahead and remove it.

Comment 33

[Philippe M. Chiasson (:gozer)]

(In reply to comment #32)
> (From update of attachment 400217 [details] [diff] [review])
> Well, I guess since Linux doesn't have a license displayed on unpack either, we
> can go ahead and remove it.

Then the patch just needs review, not sure who I should request it from. Fallen ?

Comment 34

[Philippe M. Chiasson (:gozer)]

Comment on attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

changeset:   3782:78ad69924a87
tag:         tip
user:        Philippe M. Chiasson <[e-mail]>
date:        Mon Sep 14 10:50:34 2009 -0400
summary:     Bug 510348 - Remove Sunbird license resource from the DMG on Mac OS X. r=Fallen

Comment 35

[Philippe M. Chiasson (:gozer)]

I forced a nightly this afternoon, and once it completed, l10n repacks fired proprely and succeeded.