Closed Bug 510348 Opened 10 years ago Closed 10 years ago

Sunbird Mac l10n builds failing due to requiring license unpack

Categories

(Calendar :: Build Config, defect)

All
macOS
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: standard8, Assigned: gozer)

References

Details

Attachments

(2 files, 3 obsolete files)

Attached file Log β€”
The Sunbird Mac l10n builds are failing since the switch to the new buildbot code which does l10n merge.

I'm not sure what broke this, it could be the recent switch to the new l10n code that also enables l10n merge.

Typical Log:

http://build.mozillamessaging.com/buildbot/production/builders/Sunbird%20MacOSX%2010.4%20comm-1.9.1%20sunbird%20l10n/builds/1962/steps/shell_3/logs/stdio
gah. We just found out today that Tiger's hdiutil is crappy enough to not work without manual intervention, i.e. accepting that license.

This broke on Tiger with bug 498500, and backing that out would break all SeaMonkey L10n repacks, nightlies as well as releases.
Depends on: 498500
Flags: blocking-calendar1.0?
Yeah, we need to fix this for 1.0alpha1 obviously. Stefan, I trust you to + such bugs, go ahead and do so in the future.
Flags: blocking-calendar1.0? → blocking-calendar1.0+
Depends on: 510208
Any update on this regression ? L10n builds are now missing since 7 weeks.
I've put this patch together, and with it, I've been able to run 'make unpack' successfully for a Sunbird build.

I am not sure of what the best way for a product to communicate the fact a EULA needs to be accepted. For now, setting UNPACK_ACCEPT_LICENSE in product/locale/Makefile.in does the trick.

Feedback on this approach and testing on Win32 would be appreciated.
The patch sounds wrong to me. This is not about a specific product needing to use the expect approach, it's about Tiger needing it.
(In reply to comment #5)
> The patch sounds wrong to me. This is not about a specific product needing to
> use the expect approach, it's about Tiger needing it.

Are you sure, all my development/testing so far was done on Leopard and Snow Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?

But in any case, altering when to enable this workaround will be a simple change.
(In reply to comment #6)
> Are you sure, all my development/testing so far was done on Leopard and Snow
> Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?

The Leopard and newer hdiutil doesn't forcibly ask for the license every time and doesn't need the expect script, the Tiger one does. Our problem here is that the Sunbird build machines are running on Tiger and have that even worse version of hdiutil. That tool is the problem, not the DMGs themselves.

> But in any case, altering when to enable this workaround will be a simple
> change.

True, but we should do it the right way so we can remove that hacky workaround where we desupport Tiger.
(In reply to comment #7)
> (In reply to comment #6)
> > Are you sure, all my development/testing so far was done on Leopard and Snow
> > Leopard? Looks to me like this is a proprety of the Sunbird DMGs ?
> 
> The Leopard and newer hdiutil doesn't forcibly ask for the license every time
> and doesn't need the expect script, the Tiger one does. 

That strange, I tried 'make unpack' on my Leopard box, and it got stuck asking for the license, so it might have something to do with the DMG itself, no ?

> Our problem here is
> that the Sunbird build machines are running on Tiger and have that even worse
> version of hdiutil. That tool is the problem, not the DMGs themselves.

And that shouldn't be too hard a situation to detect

> > But in any case, altering when to enable this workaround will be a simple
> > change.
> 
> True, but we should do it the right way so we can remove that hacky workaround
> where we desupport Tiger.

IMO, the right thing would be conditional on the version of hdutil instead of the version of OS X, no ?
(In reply to comment #8)
Interesting, I never saw it on Leopard boxes, but any I could test on were machines configured for our build reference platform. Maybe it depends on the version of the tools, either the ones used to generate the DMGs or to unpack them - maybe on the Xcode version used? Is hdiutil part of the OS or of Xcode?
A simpler version that simply relies on the OS X version to switch between unpack methods. Thoughts?
Attachment #398194 - Attachment is obsolete: true
Comment on attachment 398433 [details] [diff] [review]
bring back unpack's ability to accept EULAs WIP2

>-hdiutil attach -verbose -noautoopen -mountpoint $MOUNTPOINT "$DMG_PATH"
>+# Thunderbird 2.0.0.x (and others) have a license file and build on
>+# 10.4. For these, we need to use an expect script to mount them.

I think "have a license" is misleading, we probably can just say they build on 10.4 and use an older hdiutil that need a license acknowledgment, or something like that. Your comment sounds like it would be a problem with Thunderbird itself, but it's Apple who did a sucky cmdline tool. :P

Other than that, this looks fine for me.
Here is a complete version that should work for Sunbird. Ready for review.
Attachment #398433 - Attachment is obsolete: true
Is expect available on the mac box? I've seen systems where this isn't installed by default.
(In reply to comment #13)
> Is expect available on the mac box? I've seen systems where this isn't
> installed by default.

It's what we used before switching to the new script that broke you, this is only conditionally restoring the old behavior, so if that box ever worked, it has expect. I'd guess it's part of Xcode or so.
So, who should officially review this? If I were to review it, I'd give r+ codewise, but I can't test this since no mac.
Does this implies that the Sunbird/Lightning build server requires update? For compatibility reasons it probably should use the same build environment as Thunderbird.
A toolkit build system peer or the owner needs to review it.

And yes, the real issue is that the Sunbird build machine is using an older operating system version as all our other 1.9.1-based products. Not sure if the same is true for Lightning, but as it's an extension, it doesn't need to repack DMGs and doesn't run into that issue, this is Sunbird only.
I just wonder if the still expected lifetime of Sunbird warrants getting a newer build machine for it (or adding it to a newer build machine pool).
Setting up build machinery shouldn't be the first hurdle to saving Sunbird, if someone steps up.

Anyway, I think we should upgrade the lightning build machines, this way we can avoid problems that may show up if core decides to depend on 10.6 or a newer version of Xcode.
(In reply to comment #18)
> Setting up build machinery shouldn't be the first hurdle to saving Sunbird, if
> someone steps up.
> 
> Anyway, I think we should upgrade the lightning build machines, this way we can
> avoid problems that may show up if core decides to depend on 10.6 or a newer
> version of Xcode.

In that case, you'd want to upgrade the build box to 10.5. Just to be clear, Kairo, if these builds were made on 10.5, this patch would be unnecessary ?

Because, if so, upgrading the build box is another perfectly valid option, one that's also closer to what Thunderbird is doing (we build on 10.5 only now)
(In reply to comment #19)
> In that case, you'd want to upgrade the build box to 10.5. Just to be clear,
> Kairo, if these builds were made on 10.5, this patch would be unnecessary ?

Yes, upgrading the build box to 10.5 and the Xcode version we usually use on 10.5 as described in the refplatform docs should solve this problem once an for all as we have a different hdiutil version with different problems there - but for the problems of the new version we have a workaround in place already ;-)

> Because, if so, upgrading the build box is another perfectly valid option, one
> that's also closer to what Thunderbird is doing (we build on 10.5 only now)

Right, and because of that, we're not seeing this problem on Thunderbird (or Firefox or SeaMonkey, for that matter).
(In reply to comment #19)
> Because, if so, upgrading the build box is another perfectly valid option, one
> that's also closer to what Thunderbird is doing (we build on 10.5 only now)

How long would this take (including time until you have time to take care) ?
(In reply to comment #21)
> (In reply to comment #19)
> > Because, if so, upgrading the build box is another perfectly valid option, one
> > that's also closer to what Thunderbird is doing (we build on 10.5 only now)
> 
> How long would this take (including time until you have time to take care) ?

First, you'd need MoCo IT to reimage that Xserve to the 10.5 refplatform image. Once that's done, should be very quick to have it producing builds again.

Once that's done, it's probably an hour of my time to have buildbot setup on it, so I can do it pretty much anytime, except during the upcoming beta4 build period.

Best order of business is probably filing a MoCo IT bug for the re-imaging, and seeing how soon they could get to it.
Temporarly disabled l10n repack jobs for Sunbird, they are failing anyways and waste build time.
cb-xserve03 is back up and running leopard. Only issue is that some of the bits of the reference image (/tools/*) are Intel only binaries, so I'll have to rebuild these.
Summary: Sunbird Mac l10n builds failing due to requiring license unpack → Sunbird: Upgrade cb-xserve03 to OS X Leopard
cb-xserve03 has the required packages rebuilt. I just restarted buildbot on it. Should start picking up pending builds.
Back to producing Sunbird/Lightning builds, all GREEN
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
And the l10n repacks jobs fail in exactly the same way, bummer. Turns out that the Sunbird DMG have an extra license resource in the DMG itself, and I really suspect that's the culprit.

One can either use attachment 398461 [details] [diff] [review] to teach make unpack to use expect and accept the DMG license, or one can use this attachment to remove the license in question from the DMG resource. That solution feels simple to me, and nothing except Sunbird seems to be doing that. Changing issues surrounding licenses might be more complex than I can anticipate.
Attachment #398461 - Attachment is obsolete: true
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Interesting. i totally agree that making Sunbird the same as everybody else is the best solution for this.
Did other used to have this maybe? We don't have the about:rights bar, not sure if this has legal consequences?
SeaMonkey also doesn't have about:rights (yet), and I can't remember hearing anything about anyone else having this or needing it for legal consequences, but it might have been there for someone in some past - not since 1.9.* though or we would have caught it with using the same unpack scripts all over various places (L10n repackaging, release package verification in buildbot, and apparently talos).
Comment on attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

Well, I guess since Linux doesn't have a license displayed on unpack either, we can go ahead and remove it.
Attachment #400217 - Flags: review+
(In reply to comment #32)
> (From update of attachment 400217 [details] [diff] [review])
> Well, I guess since Linux doesn't have a license displayed on unpack either, we
> can go ahead and remove it.

Then the patch just needs review, not sure who I should request it from. Fallen ?
Comment on attachment 400217 [details] [diff] [review]
[checked in] Remove license resource from the DMG

changeset:   3782:78ad69924a87
tag:         tip
user:        Philippe M. Chiasson <gozer@mozillamessaging.com>
date:        Mon Sep 14 10:50:34 2009 -0400
summary:     Bug 510348 - Remove Sunbird license resource from the DMG on Mac OS X. r=Fallen
Attachment #400217 - Attachment description: Remove license resource from the DMG → [checked in] Remove license resource from the DMG
I forced a nightly this afternoon, and once it completed, l10n repacks fired proprely and succeeded.
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Summary: Sunbird: Upgrade cb-xserve03 to OS X Leopard → Sunbird Mac l10n builds failing due to requiring license unpack
Assignee: nobody → gozer
You need to log in before you can comment on or make changes to this bug.