Closed Bug 661910 Opened 11 years ago Closed 11 years ago

Windows XULRunner Aurora 6 SDK is missing

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(firefox5 unaffected, firefox6+ fixed, firefox7 fixed, firefox8 fixed)

Status:
VERIFIED FIXED
Milestone:
mozilla6
Tracking Flags:
Tracking Status
firefox5 --- unaffected
firefox6 + fixed
firefox7 --- fixed
firefox8 --- fixed

People

(Reporter: WeirdAl, Assigned: glandium)

References

(
URL
)

Details

Attachments

(1 file, 2 obsolete files)

Iterate over all component manifests when commenting out binary components during omnijar packing
2.30 KB, patch
khuey
: review+
asa
: approval-mozilla-aurora+
asa
: approval-mozilla-beta+
Details | Diff | Splinter Review 
The company I work for requires a Windows SDK to compile a binary component for FF6.  Currently, the latest-mozilla-aurora directory for XULRunner only has Linux builds.  Please advise!
A recent build failed in make package with:
updater.exe
xpcom.dll
xpcshell.exe
xpidl.exe
xul.dll
Linking XPT files...
xulrunner-stub.exe
xulrunner.exe
d:/mozilla-build/python25/python2.5.exe /e/builds/moz2_slave/cen-w32-xr/build/config/optimizejars.py --optimize /e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer/../../jarlog/ ../../dist/bin/chrome ../../dist/xulrunner/chrome
Removing unpackaged files...
cd ../../dist/xulrunner; rm -rf xulrunner-config regchrome* regxpcom*  core bsdecho gtscc js js-config jscpucfg nsinstall viewer TestGtkEmbed codesighs* elf-dynstr-gc mangle* maptsv* mfc* mkdepend* msdump* msmap* nm2tsv* nsinstall* res/samples res/throbber shlibsign* ssltunnel* certutil* pk12util* winEmbed.exe chrome/chrome.rdf chrome/app-chrome.manifest chrome/overlayinfo components/compreg.dat components/xpti.dat content_unit_tests necko_unit_tests *.dSYM 
e:/builds/moz2_slave/cen-w32-xr/build/obj-firefox/config/nsinstall.exe -D ../../dist/
Compressing...
cd ../../dist && (cd xulrunner && rm -f omni.jar components/binary.manifest && grep -h '^binary-component' components/*.manifest > binary.manifest ; sed -e 's/^binary-component/#binary-component/' components/components.manifest > components.manifest && mv components.manifest components && zip -r9m omni.jar chrome chrome.manifest components/*.js components/*.xpt components/*.manifest modules res defaults greprefs.js jsloader  -x chrome/icons/\* defaults/pref/channel-prefs.js res/cursors/\* res/MainMenu.nib/\*  && true && d:/mozilla-build/python25/python2.5.exe /e/builds/moz2_slave/cen-w32-xr/build/config/optimizejars.py --optimize /e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer/../../jarlog/ ./ ./ && mv binary.manifest components && printf "manifest components/binary.manifest\n" > chrome.manifest) && (cd xulrunner && d:/mozilla-build/python25/python2.5.exe /e/builds/moz2_slave/cen-w32-xr/build/config/createprecomplete.py) && zip -r9D xulrunner-7.0a1.en-US.win32.zip xulrunner
make[2]: Leaving directory `/e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer'
make[1]: Leaving directory `/e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer'
sed: can't read components/components.manifest: No such file or directory
make[2]: *** [make-package] Error 2
make[1]: *** [all] Error 2
make: *** [package] Error 2
Took a quick look and in this instance it appears to be related to there being no components in the components directory after packaging on Windows unlike bug 653971 where on Mac the package has one binary component in the components directory. I'll look a bit more later.
Attached patch patch in progress (obsolete) — — Splinter Review 
Not sure this is the right approach... just putting this in the bug in case someone has time to look at this which I don't at the moment.
Mike, fairly certain switching xulrunner to omnijar caused this bug / broke Win32 xulrunner builds.
Blocks: 620931
I wonder if this is related to bug 661792.  If it's the same problem, this should be fixed during the buildbot master reconfigure tomorrow.  If not, something else needs fixing.
I saw that and think it might be related but it might not be fixed by it due to xulrunner not having any binary components on Windows.
An easy immediate fix would be to backout part 5 of bug 620931 (https://bug620931.bugzilla.mozilla.org/attachment.cgi?id=499282 )

That being said, I think the problem is that xulrunner doesn't have a package manifest, and as such, components manifests are not aggregated. I think it's about time we have a package manifest for the toolkit, that the browser package manifest would include. And then (finally) have one for xulrunner.
(In reply to comment #7)
> That being said, I think the problem is that xulrunner doesn't have a
> package manifest, and as such, components manifests are not aggregated.

meaning we have plenty of components/*.manifest files, and no components/components.manifest nor components/interfaces.manifest.
(In reply to comment #7)
> I think it's about time we have a package manifest for the toolkit, that the
> browser package manifest would include. And then (finally) have one for
> xulrunner.

That wouldn't solve the problem for other things not using a package manifest, though. So an easy first way out would be to iterate over components/*.manifest instead of dealing with components/components.manifest only.
Attachment #537689 - Attachment is obsolete: true
Based on the patch, looks like this belongs in Core: Build Config
Component: Release Engineering → Build Config
Product: mozilla.org → Core
QA Contact: release → build-config
Version: other → Trunk
Could someone at least push this to tryserver, see if it fixes the build?
Pushing it to tryserver isn't going to kick off a XULRunner SDK build ...
(In reply to comment #7)
> That being said, I think the problem is that xulrunner doesn't have a
> package manifest, and as such, components manifests are not aggregated. I
> think it's about time we have a package manifest for the toolkit, that the
> browser package manifest would include. And then (finally) have one for
> xulrunner.

bug 526333
I've tested the patch locally, and it works.
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Comment 15 says this works locally (the best test we have for XR SDK). Seems fine to me, but sending to khuey for sanity checks.
Attachment #537710 - Flags: review?(khuey)
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Looks reasonable enough, and one of the ten people on earth who care about XULRunner says it works, so let's do it.
Attachment #537710 - Flags: review?(khuey) → review+
FWIW, mozilla-central XULRunner builds are broken for the same reason, according to Tinderbox, so we need to land this patch there too.
http://hg.mozilla.org/mozilla-central/rev/5bb18a13d2b9

Going to wait for tomorrow's XULRunner builds before closing this out.
XULRunner on m-c completed successfully this morning.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

XULRunner tinderbox shows this fixes the build on mozilla-central.  Requesting approval for Aurora.
Attachment #537710 - Flags: approval-mozilla-aurora?
This may have broken Mac XULRunner builds :-(
I'm going to back it out to see.
Backing this out fixed Mac XULRunner builds, so we'll need a new patch here.
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Clearing approval request because this patch has unacceptable side effects.
Attachment #537710 - Flags: approval-mozilla-aurora?
For the record, the mac failure log is here:
http://tinderbox.mozilla.org/showlog.cgi?tree=XULRunner&errorparser=unix&logfile=1307959643.1307967855.17882.gz&buildtime=1307959643&buildname=OS%20X%2010.6.2%20mozilla-central%20xulrunner&fulltext=1

The relevant part:
/builds/slave/cen-osx64-xr/build/build/macosx/universal/unify \
          --unify-with-sort "\.manifest$" \
          --unify-with-sort "components\.list$" \
	  obj-firefox/i386/dist/xulrunner/XUL.framework \
	  obj-firefox/x86_64/dist/xulrunner/XUL.framework \
	  obj-firefox/i386/dist/universal/xulrunner/XUL.framework
/builds/slave/cen-osx64-xr/build/build/macosx/universal/unify: warning: makeUniversalDirectory: only in x86 obj-firefox/x86_64/dist/xulrunner/XUL.framework/Versions/7.0a1:
  xulrunner
Can't call method "path" on an undefined value at /builds/slave/cen-osx64-xr/build/build/macosx/universal/unify line 1092.
make[2]: *** [postflight_all] Error 255
make[1]: *** [realbuild] Error 2

There effectively no xulrunner file in one of the zip outputs, but I don't see how this relates to this patch...
(In reply to comment #27)
> There effectively no xulrunner file in one of the zip outputs, but I don't
> see how this relates to this patch...

Not zip, rsync.
I just tried build XULRunner SDK on my Mac with the patch applied (i386 only), and it worked fine.  I can't build x86_64 as the MacBook is not a 64-bit machine (probably about 5 years old).
Duplicate of this bug: 666980
(In reply to comment #28)
> (In reply to comment #27)
> > There effectively no xulrunner file in one of the zip outputs, but I don't
> > see how this relates to this patch...
> 
> Not zip, rsync.

That's bug 667012.
This busted win32 xulrunner builds for 6.0 beta.
The last time we saw a win32 xulrunner was June 13th
http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-mozilla-central/?C=M;O=A
http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-mozilla-aurora/?C=M;O=D


I see the following files for Mac (dated July 11th):
* xulrunner-8.0a1.en-US.mac-pkg.dmg
* xulrunner-8.0a1.en-US.mac-i386.sdk.tar.bz2
* xulrunner-8.0a1.en-US.mac-pkg.txt
* xulrunner-8.0a1.en-US.mac-x86_64.sdk.tar.bz2
* xulrunner-8.0a1.en-US.mac-pkg.checksums

I believe this got fixed for Mac with khuey's change:
http://hg.mozilla.org/mozilla-central/rev/518a79e90444cce2919b4e13a559b45aef080360

Does this mean that this is fixed for Mac but still broken for Win32?
Could we separate the two issues?

Does anyone know what the issue for Windows is? It seems that most comments are with regards to the Mac issue.
The fix for Windows broke Mac, so we backed it out.

I'll look into this if I ever get the Mac from Bug 664011.  Been waiting a month now ...
(In reply to comment #34)
> The fix for Windows broke Mac, so we backed it out.
> 
> I'll look into this if I ever get the Mac from Bug 664011.  Been waiting a
> month now ...

I believe you fixed the issue for Mac on your push this weekend.
Do you need the Mac to run Windows on it? (I might be getting things mixed up).

Now I understand why we have xulrunner builds on the 13th since a fix got landed and backed out.


On another note, I believe xulrunner nightly builds should show up on Firefox's tbpl trees (m-c, m-a & m-b). Anyone disagrees? IMHO it would help catch issues earlier.
(In reply to comment #35)
> (In reply to comment #34)
> > The fix for Windows broke Mac, so we backed it out.
> > 
> > I'll look into this if I ever get the Mac from Bug 664011.  Been waiting a
> > month now ...
> 
> I believe you fixed the issue for Mac on your push this weekend.
> Do you need the Mac to run Windows on it? (I might be getting things mixed
> up).

Mac should have been fixed on June 13th or whenever the backout happened.  I need a Mac build environment to see why this change broke Mac.

> Now I understand why we have xulrunner builds on the 13th since a fix got
> landed and backed out.

Right.

> On another note, I believe xulrunner nightly builds should show up on
> Firefox's tbpl trees (m-c, m-a & m-b). Anyone disagrees? IMHO it would help
> catch issues earlier.

I don't disagree, but this is something that should be raised more widely.  I'll bring it up at the engineering meeting tomorrow.
Sigh, apparently the Mac XR builds randomly turn red ... and it looks like landing this patch coincided with that.
And this was random XR mac bustage.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Drivers, we will need to take this on Aurora and Beta so that third parties can build binary addons.
Attachment #537710 - Flags: approval-mozilla-beta?
Attachment #537710 - Flags: approval-mozilla-aurora?
Filed Bug 670951 for the intermittent Mac failures.
Alex, can you verify that the SDKs you need are all on present on FTP for mozilla-central?
Confirmed: ftp://ftp.mozilla.org/pub/xulrunner/nightly/latest-mozilla-central/xulrunner-8.0a1.en-US.win32.zip is there for me to pull, if that were the one I wanted.  (We're sticking to Aurora builds.  :-) )

Also, tinderbox shows XULRunner on mozilla-central building that .sdk.zip file.  That would be the one I need from Aurora, and now Beta.
Yeah, I'll get it on the branches asap.
Attachment #537710 - Flags: approval-mozilla-beta?
Attachment #537710 - Flags: approval-mozilla-beta+
Attachment #537710 - Flags: approval-mozilla-aurora?
Attachment #537710 - Flags: approval-mozilla-aurora+
:-D I now have SDK's for FF6 Beta and FF7 Aurora from ftp.mozilla.org - thanks!
Yay!
Status: RESOLVED → VERIFIED
Assignee: nobody → mh+mozilla
This seems to have landed on Aurora, so we didn't transplant it to beta during migration:

http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a
(In reply to Ehsan Akhgari [:ehsan] from comment #48)
> This seems to have landed on Aurora, so we didn't transplant it to beta
> during migration:
> 
> http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a

What does that mean?  It came along with the merge, no?
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #49)
> (In reply to Ehsan Akhgari [:ehsan] from comment #48)
> > This seems to have landed on Aurora, so we didn't transplant it to beta
> > during migration:
> > 
> > http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a
> 
> What does that mean?  It came along with the merge, no?

No, it was landed separately by you: http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?changeset=e0a0e1800c6a

I just wanted to mention that this change did not make it into Beta.  If you think it should, please nominate it.  Thanks!
(In reply to Ehsan Akhgari [:ehsan] from comment #50)
> (In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #49)
> > (In reply to Ehsan Akhgari [:ehsan] from comment #48)
> > > This seems to have landed on Aurora, so we didn't transplant it to beta
> > > during migration:
> > > 
> > > http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a
> > 
> > What does that mean?  It came along with the merge, no?
> 
> No, it was landed separately by you:
> http://hg.mozilla.org/releases/mozilla-aurora/
> pushloghtml?changeset=e0a0e1800c6a
> 
> I just wanted to mention that this change did not make it into Beta.  If you
> think it should, please nominate it.  Thanks!

http://hg.mozilla.org/releases/mozilla-beta/rev/e0a0e1800c6a

Sure looks like it's in beta to me ...
Hmm, then I don't remember why I commented here during the migration, but there _was_ a reason... :/
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.