Closed Bug 410637 Opened 13 years ago Closed 13 years ago

Mac UB XULRunner SDK packages have empty bin directory

Categories

(Firefox Build System :: General, defect)

PowerPC
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla1.9beta5

People

(Reporter: matthew.gertner, Assigned: mossop)

References

Details

Attachments

(3 files, 1 obsolete file)

I downloaded a XULRunner SDK nightly for Mac from the FTP server (specifically http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-trunk/xulrunner-1.9b3pre.en-US.mac-pkg.sdk.tar.bz2). When I unpacked it, the bin directory under the root was empty.
Also, this may or may not be related, but I get the following error if I copy over the bin/ files manually:

/usr/bin/ld: warning /Users/Matt/xulrunner-sdk/lib/libxpcomglue_s.a archive's cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (can't load from it)

Maybe the libraries aren't i386 compatible?
Also XUL.framework doesn't appear to be in there.
We recently had to change build machines for XULRunner due to a hardware failure. It's quite possible that there is something wrong with the configuration of the new machine.

Do you have a regression range for the when the bin/ directory was last populated correctly?
I don't personally, no. The first SDK I ever downloaded is referenced in https://bugzilla.mozilla.org/show_bug.cgi?id=410637#c0.
(In reply to comment #4)
> I don't personally, no. The first SDK I ever downloaded is referenced in
> https://bugzilla.mozilla.org/show_bug.cgi?id=410637#c0.

That link points to the latest-trunk directory which gets overwritten with the latest build every night. One of the date-stamped directories under http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/ is what I was hoping for.

The oldest Mac SDK nightly on ftp right now is from 2007-11-22, and it's bin/ dir is empty as well. This looks like a larger packaging problem.
Assignee: nobody → matthew
I'm going to take a crack at this. If anyone has ideas about what the problem is, please let me know.
packager.mk is used for several purposes. It looks like the part that copies dist/bin/ into the staging directory for packaging into the SDK is broken. There is something there to copy _APPNAME into the staging directory when UNIVERSAL_BINARY is not set, and since when building the SDK _APPNAME is set to something weird, I'm actually not sure why the build doesn't simply break when the UB make files try to make the SDK for i386 and ppc without UNIVERSAL_BINARY set.

Nonetheless, here's my first crack at a solution. Can someone tell me exactly how the tinderbox version is built so I can test locally? Building the SDK individually in i386 and ppc now works fine (as long as UNIVERSAL_BINARY is not set), but this isn't entirely reassuring since, as I mention, I don't understand how the build is even completing right now.
Attachment #304951 - Flags: review?(benjamin)
Attachment #304951 - Attachment is patch: true
Attachment #304951 - Attachment mime type: application/octet-stream → text/plain
Attachment #304951 - Flags: review?(benjamin) → review+
Comment on attachment 304951 [details] [diff] [review]
Copy dist/bin/ directory into staging directory when building SDK

Shouldn't affect anything except the Mac XR SDK.
Attachment #304951 - Flags: approval1.9?
Comment on attachment 304951 [details] [diff] [review]
Copy dist/bin/ directory into staging directory when building SDK

a1.9=beltzner
Attachment #304951 - Flags: approval1.9? → approval1.9+
Keywords: checkin-needed
Checking in toolkit/mozapps/installer/packager.mk;
/cvsroot/mozilla/toolkit/mozapps/installer/packager.mk,v  <--  packager.mk
new revision: 1.54; previous revision: 1.53
done
Status: NEW → RESOLVED
Closed: 13 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9beta5
The nightly sdks still have an empty bin directory, reopening
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Duplicate of this bug: 418596
Attached patch Fix sdk packaging in UB setup (obsolete) — Splinter Review
This is probably not quite ready yet and I haven't tested that other apps still build right with it.

This simply unsets UNIVERSAL_BINARY for the stage-package pass so we properly copy stuff into sdk-stage. This will let the tinderbox correctly build either a ppc or an i386 sdk after a UB xulrunner build. Might want to modify the tarball name for this.

I've also added in dist/host/bin to the sdk under host/bin. It is here that xpidl is found for the ppc pass as host_xpidl so I think it is necessary. Perhaps some of the other things aren't there though I'm not sure.

flight.mk contains a couple of tweaks to make it work right for apps built against the sdk. It assumes they have built themselves an app bundle and copied the <sdk>/bin into Contents/Frameworks/XUL.Framework of the bundle. postflight will then fix the buildconfig and clear out the .chk files and unify correctly.

This patch and a couple of lines changed in McCoy's makefiles were all that were necessary to make a UB of McCoy against the two resulting sdks.
My hero!
Assignee: matthew → dtownsend
Status: REOPENED → NEW
Two parts to this patch, this is to the main build system.

flight.mk is fixed to attempt to find the toolkit.jar file and the nss chk files regardless of the build type.

The sdk package name is changed on mac to include the processor. I couldn't find a better way to do this but it generates names like:

xulrunner-1.9pre.en-US.mac-i386.sdk.tar.bz2
xulrunner-1.9pre.en-US.mac-powerpc.sdk.tar.bz2

rather than the current:

xulrunner-1.9pre.en-US.mac-pkg.sdk.tar.bz2

The dist/host/bin directory is included in the sdk such that the powerpc sdk includes host/bin/xpidl in it.

The sdk is no longer build at the same time as the installer. The installer will build a UB version of xulrunner, then separately we must build the ppc and i386 sdks. I couldn't see an easier way to do that. Next patch makes tinderbox perform that additional step.
Attachment #313498 - Attachment is obsolete: true
Attachment #314347 - Flags: review?(benjamin)
This adds code to tinderbox so that if configured it will perform an sdk build for the objdir, or for both objdirs in a UB case.
Attachment #314347 - Flags: review?(benjamin) → review+
Comment on attachment 314348 [details] [diff] [review]
Make tinderbox build sdk

This adds an additional build step post build which makes the sdk and copies it to the packages directory. In a unversal build case it repeats this step for the i386 objdir.
Attachment #314348 - Flags: review?(nrthomas)
Comment on attachment 314348 [details] [diff] [review]
Make tinderbox build sdk

A bit painful to be duplicating crappy code, but at least it's self contained.
Attachment #314348 - Flags: review?(nrthomas) → review+
Comment on attachment 314347 [details] [diff] [review]
Fix sdk packaging

Requesting approval to land this for 1.9. This makes it possible to generate working gecko sdks for Mac. The only impact to Firefox is some changes to the makefiles that performs the final unification on Mac. There should be no chance of any hidden regressions from this, if t were wrong the Firefox build would just fail. I have of course tested this myself and seen no issue however.
Attachment #314347 - Flags: approval1.9?
Comment on attachment 314347 [details] [diff] [review]
Fix sdk packaging

a1.9=beltzner
Attachment #314347 - Flags: approval1.9? → approval1.9+
Landed

Checking in build/macosx/universal/flight.mk;
/cvsroot/mozilla/build/macosx/universal/flight.mk,v  <--  flight.mk
new revision: 1.9; previous revision: 1.8
done
Checking in toolkit/mozapps/installer/packager.mk;
/cvsroot/mozilla/toolkit/mozapps/installer/packager.mk,v  <--  packager.mk
new revision: 1.58; previous revision: 1.57
done
Checking in xulrunner/installer/Makefile.in;
/cvsroot/mozilla/xulrunner/installer/Makefile.in,v  <--  Makefile.in
new revision: 1.15; previous revision: 1.14
done

Checking in tinderbox/post-mozilla-rel.pl;
/cvsroot/mozilla/tools/tinderbox/post-mozilla-rel.pl,v  <--  post-mozilla-rel.pl
new revision: 1.145; previous revision: 1.144
done
Checking in tinderbox/tinder-defaults.pl;
/cvsroot/mozilla/tools/tinderbox/tinder-defaults.pl,v  <--  tinder-defaults.pl
new revision: 1.127; previous revision: 1.126
done
Checking in tinderbox-configs/xulrunner/linux/tinder-config.pl;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/linux/tinder-config.pl,v  <--  tinder-config.pl
new revision: 1.9; previous revision: 1.8
done
Checking in tinderbox-configs/xulrunner/macosx/tinder-config.pl;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/macosx/tinder-config.pl,v  <--  tinder-config.pl
new revision: 1.7; previous revision: 1.6
done
Checking in tinderbox-configs/xulrunner/win32/tinder-config.pl;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/win32/tinder-config.pl,v  <--  tinder-config.pl
new revision: 1.12; previous revision: 1.11
done
Status: NEW → RESOLVED
Closed: 13 years ago13 years ago
Resolution: --- → FIXED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.