Last Comment Bug 661910 - Windows XULRunner Aurora 6 SDK is missing
: Windows XULRunner Aurora 6 SDK is missing
Status: VERIFIED FIXED
:
Product: Core
Classification: Components
Component: Build Config (show other bugs)
: Trunk
: x86 Windows 7
: -- major (vote)
: mozilla6
Assigned To: Mike Hommey [:glandium]
:
Mentors:
ftp://ftp.mozilla.org/pub/mozilla.org...
: 666980 (view as bug list)
Depends on: 664011
Blocks: tracking_win64 575912 620931
  Show dependency treegraph
 
Reported: 2011-06-03 12:27 PDT by Alex Vincent [:WeirdAl]
Modified: 2013-12-27 14:34 PST (History)
11 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
unaffected
+
fixed
fixed
fixed


Attachments
patch in progress (2.32 KB, patch)
2011-06-06 17:01 PDT, Robert Strong [:rstrong] (use needinfo to contact me)
no flags Details | Diff | Splinter Review
Iterate over all component manifests when commenting out binary components during omnijar packing (2.30 KB, patch)
2011-06-06 18:49 PDT, Mike Hommey [:glandium]
khuey: review+
asa: approval‑mozilla‑aurora+
asa: approval‑mozilla‑beta+
Details | Diff | Splinter Review
Back out since it may have broken Mac XULRunner builds. (2.31 KB, patch)
2011-06-13 11:26 PDT, Josh Matthews [:jdm]
no flags Details | Diff | Splinter Review

Description Alex Vincent [:WeirdAl] 2011-06-03 12:27:13 PDT
The company I work for requires a Windows SDK to compile a binary component for FF6.  Currently, the latest-mozilla-aurora directory for XULRunner only has Linux builds.  Please advise!
Comment 1 Ben Hearsum (:bhearsum) 2011-06-03 12:28:30 PDT
A recent build failed in make package with:
updater.exe
xpcom.dll
xpcshell.exe
xpidl.exe
xul.dll
Linking XPT files...
xulrunner-stub.exe
xulrunner.exe
d:/mozilla-build/python25/python2.5.exe /e/builds/moz2_slave/cen-w32-xr/build/config/optimizejars.py --optimize /e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer/../../jarlog/ ../../dist/bin/chrome ../../dist/xulrunner/chrome
Removing unpackaged files...
cd ../../dist/xulrunner; rm -rf xulrunner-config regchrome* regxpcom*  core bsdecho gtscc js js-config jscpucfg nsinstall viewer TestGtkEmbed codesighs* elf-dynstr-gc mangle* maptsv* mfc* mkdepend* msdump* msmap* nm2tsv* nsinstall* res/samples res/throbber shlibsign* ssltunnel* certutil* pk12util* winEmbed.exe chrome/chrome.rdf chrome/app-chrome.manifest chrome/overlayinfo components/compreg.dat components/xpti.dat content_unit_tests necko_unit_tests *.dSYM 
e:/builds/moz2_slave/cen-w32-xr/build/obj-firefox/config/nsinstall.exe -D ../../dist/
Compressing...
cd ../../dist && (cd xulrunner && rm -f omni.jar components/binary.manifest && grep -h '^binary-component' components/*.manifest > binary.manifest ; sed -e 's/^binary-component/#binary-component/' components/components.manifest > components.manifest && mv components.manifest components && zip -r9m omni.jar chrome chrome.manifest components/*.js components/*.xpt components/*.manifest modules res defaults greprefs.js jsloader  -x chrome/icons/\* defaults/pref/channel-prefs.js res/cursors/\* res/MainMenu.nib/\*  && true && d:/mozilla-build/python25/python2.5.exe /e/builds/moz2_slave/cen-w32-xr/build/config/optimizejars.py --optimize /e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer/../../jarlog/ ./ ./ && mv binary.manifest components && printf "manifest components/binary.manifest\n" > chrome.manifest) && (cd xulrunner && d:/mozilla-build/python25/python2.5.exe /e/builds/moz2_slave/cen-w32-xr/build/config/createprecomplete.py) && zip -r9D xulrunner-7.0a1.en-US.win32.zip xulrunner
make[2]: Leaving directory `/e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer'
make[1]: Leaving directory `/e/builds/moz2_slave/cen-w32-xr/build/obj-firefox/xulrunner/installer'
sed: can't read components/components.manifest: No such file or directory
make[2]: *** [make-package] Error 2
make[1]: *** [all] Error 2
make: *** [package] Error 2
Comment 2 Robert Strong [:rstrong] (use needinfo to contact me) 2011-06-03 22:52:05 PDT
Took a quick look and in this instance it appears to be related to there being no components in the components directory after packaging on Windows unlike bug 653971 where on Mac the package has one binary component in the components directory. I'll look a bit more later.
Comment 3 Robert Strong [:rstrong] (use needinfo to contact me) 2011-06-06 17:01:20 PDT
Created attachment 537689 [details] [diff] [review]
patch in progress

Not sure this is the right approach... just putting this in the bug in case someone has time to look at this which I don't at the moment.
Comment 4 Robert Strong [:rstrong] (use needinfo to contact me) 2011-06-06 17:04:33 PDT
Mike, fairly certain switching xulrunner to omnijar caused this bug / broke Win32 xulrunner builds.
Comment 5 Aki Sasaki [:aki] 2011-06-06 18:19:39 PDT
I wonder if this is related to bug 661792.  If it's the same problem, this should be fixed during the buildbot master reconfigure tomorrow.  If not, something else needs fixing.
Comment 6 Robert Strong [:rstrong] (use needinfo to contact me) 2011-06-06 18:23:28 PDT
I saw that and think it might be related but it might not be fixed by it due to xulrunner not having any binary components on Windows.
Comment 7 Mike Hommey [:glandium] 2011-06-06 18:36:19 PDT
An easy immediate fix would be to backout part 5 of bug 620931 (https://bug620931.bugzilla.mozilla.org/attachment.cgi?id=499282 )

That being said, I think the problem is that xulrunner doesn't have a package manifest, and as such, components manifests are not aggregated. I think it's about time we have a package manifest for the toolkit, that the browser package manifest would include. And then (finally) have one for xulrunner.
Comment 8 Mike Hommey [:glandium] 2011-06-06 18:37:29 PDT
(In reply to comment #7)
> That being said, I think the problem is that xulrunner doesn't have a
> package manifest, and as such, components manifests are not aggregated.

meaning we have plenty of components/*.manifest files, and no components/components.manifest nor components/interfaces.manifest.
Comment 9 Mike Hommey [:glandium] 2011-06-06 18:43:16 PDT
(In reply to comment #7)
> I think it's about time we have a package manifest for the toolkit, that the
> browser package manifest would include. And then (finally) have one for
> xulrunner.

That wouldn't solve the problem for other things not using a package manifest, though. So an easy first way out would be to iterate over components/*.manifest instead of dealing with components/components.manifest only.
Comment 10 Mike Hommey [:glandium] 2011-06-06 18:49:07 PDT
Created attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Totally untested, but that should look like this.
Comment 11 Ben Hearsum (:bhearsum) 2011-06-08 07:51:56 PDT
Based on the patch, looks like this belongs in Core: Build Config
Comment 12 Alex Vincent [:WeirdAl] 2011-06-08 17:33:10 PDT
Could someone at least push this to tryserver, see if it fixes the build?
Comment 13 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-08 17:34:31 PDT
Pushing it to tryserver isn't going to kick off a XULRunner SDK build ...
Comment 14 Michael Wu [:mwu] 2011-06-09 02:41:45 PDT
(In reply to comment #7)
> That being said, I think the problem is that xulrunner doesn't have a
> package manifest, and as such, components manifests are not aggregated. I
> think it's about time we have a package manifest for the toolkit, that the
> browser package manifest would include. And then (finally) have one for
> xulrunner.

bug 526333
Comment 15 Alex Vincent [:WeirdAl] 2011-06-09 13:24:43 PDT
I've tested the patch locally, and it works.
Comment 16 Mark Finkle (:mfinkle) (use needinfo?) 2011-06-09 14:24:00 PDT
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Comment 15 says this works locally (the best test we have for XR SDK). Seems fine to me, but sending to khuey for sanity checks.
Comment 17 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-09 17:00:45 PDT
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Looks reasonable enough, and one of the ten people on earth who care about XULRunner says it works, so let's do it.
Comment 18 Alex Vincent [:WeirdAl] 2011-06-09 17:15:31 PDT
FWIW, mozilla-central XULRunner builds are broken for the same reason, according to Tinderbox, so we need to land this patch there too.
Comment 19 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-11 12:30:20 PDT
http://hg.mozilla.org/mozilla-central/rev/5bb18a13d2b9

Going to wait for tomorrow's XULRunner builds before closing this out.
Comment 20 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-12 09:33:42 PDT
XULRunner on m-c completed successfully this morning.
Comment 21 Alex Vincent [:WeirdAl] 2011-06-12 20:41:58 PDT
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

XULRunner tinderbox shows this fixes the build on mozilla-central.  Requesting approval for Aurora.
Comment 22 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-13 08:59:01 PDT
This may have broken Mac XULRunner builds :-(
I'm going to back it out to see.
Comment 23 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-13 11:01:30 PDT
http://hg.mozilla.org/mozilla-central/rev/17219648b629
Comment 24 Josh Matthews [:jdm] 2011-06-13 11:26:59 PDT
Created attachment 538960 [details] [diff] [review]
Back out  since it may have broken Mac XULRunner builds.
Comment 25 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-13 16:03:16 PDT
Backing this out fixed Mac XULRunner builds, so we'll need a new patch here.
Comment 26 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-06-13 16:03:40 PDT
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Clearing approval request because this patch has unacceptable side effects.
Comment 27 Mike Hommey [:glandium] 2011-06-15 02:37:21 PDT
For the record, the mac failure log is here:
http://tinderbox.mozilla.org/showlog.cgi?tree=XULRunner&errorparser=unix&logfile=1307959643.1307967855.17882.gz&buildtime=1307959643&buildname=OS%20X%2010.6.2%20mozilla-central%20xulrunner&fulltext=1

The relevant part:
/builds/slave/cen-osx64-xr/build/build/macosx/universal/unify \
          --unify-with-sort "\.manifest$" \
          --unify-with-sort "components\.list$" \
	  obj-firefox/i386/dist/xulrunner/XUL.framework \
	  obj-firefox/x86_64/dist/xulrunner/XUL.framework \
	  obj-firefox/i386/dist/universal/xulrunner/XUL.framework
/builds/slave/cen-osx64-xr/build/build/macosx/universal/unify: warning: makeUniversalDirectory: only in x86 obj-firefox/x86_64/dist/xulrunner/XUL.framework/Versions/7.0a1:
  xulrunner
Can't call method "path" on an undefined value at /builds/slave/cen-osx64-xr/build/build/macosx/universal/unify line 1092.
make[2]: *** [postflight_all] Error 255
make[1]: *** [realbuild] Error 2

There effectively no xulrunner file in one of the zip outputs, but I don't see how this relates to this patch...
Comment 28 Mike Hommey [:glandium] 2011-06-15 02:38:10 PDT
(In reply to comment #27)
> There effectively no xulrunner file in one of the zip outputs, but I don't
> see how this relates to this patch...

Not zip, rsync.
Comment 29 Alex Vincent [:WeirdAl] 2011-06-22 12:06:51 PDT
I just tried build XULRunner SDK on my Mac with the patch applied (i386 only), and it worked fine.  I can't build x86_64 as the MacBook is not a 64-bit machine (probably about 5 years old).
Comment 30 Benjamin Smedberg AWAY UNTIL 2-AUG-2016 [:bsmedberg] 2011-06-24 11:03:27 PDT
*** Bug 666980 has been marked as a duplicate of this bug. ***
Comment 31 Chris AtLee [:catlee] 2011-07-06 06:01:25 PDT
(In reply to comment #28)
> (In reply to comment #27)
> > There effectively no xulrunner file in one of the zip outputs, but I don't
> > see how this relates to this patch...
> 
> Not zip, rsync.

That's bug 667012.
Comment 32 Chris AtLee [:catlee] 2011-07-06 06:02:10 PDT
This busted win32 xulrunner builds for 6.0 beta.
Comment 33 Armen Zambrano [:armenzg] - Engineering productivity 2011-07-11 08:54:14 PDT
The last time we saw a win32 xulrunner was June 13th
http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-mozilla-central/?C=M;O=A
http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-mozilla-aurora/?C=M;O=D


I see the following files for Mac (dated July 11th):
* xulrunner-8.0a1.en-US.mac-pkg.dmg
* xulrunner-8.0a1.en-US.mac-i386.sdk.tar.bz2
* xulrunner-8.0a1.en-US.mac-pkg.txt
* xulrunner-8.0a1.en-US.mac-x86_64.sdk.tar.bz2
* xulrunner-8.0a1.en-US.mac-pkg.checksums

I believe this got fixed for Mac with khuey's change:
http://hg.mozilla.org/mozilla-central/rev/518a79e90444cce2919b4e13a559b45aef080360

Does this mean that this is fixed for Mac but still broken for Win32?
Could we separate the two issues?

Does anyone know what the issue for Windows is? It seems that most comments are with regards to the Mac issue.
Comment 34 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-11 08:58:32 PDT
The fix for Windows broke Mac, so we backed it out.

I'll look into this if I ever get the Mac from Bug 664011.  Been waiting a month now ...
Comment 35 Armen Zambrano [:armenzg] - Engineering productivity 2011-07-11 09:05:08 PDT
(In reply to comment #34)
> The fix for Windows broke Mac, so we backed it out.
> 
> I'll look into this if I ever get the Mac from Bug 664011.  Been waiting a
> month now ...

I believe you fixed the issue for Mac on your push this weekend.
Do you need the Mac to run Windows on it? (I might be getting things mixed up).

Now I understand why we have xulrunner builds on the 13th since a fix got landed and backed out.


On another note, I believe xulrunner nightly builds should show up on Firefox's tbpl trees (m-c, m-a & m-b). Anyone disagrees? IMHO it would help catch issues earlier.
Comment 36 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-11 09:50:53 PDT
(In reply to comment #35)
> (In reply to comment #34)
> > The fix for Windows broke Mac, so we backed it out.
> > 
> > I'll look into this if I ever get the Mac from Bug 664011.  Been waiting a
> > month now ...
> 
> I believe you fixed the issue for Mac on your push this weekend.
> Do you need the Mac to run Windows on it? (I might be getting things mixed
> up).

Mac should have been fixed on June 13th or whenever the backout happened.  I need a Mac build environment to see why this change broke Mac.

> Now I understand why we have xulrunner builds on the 13th since a fix got
> landed and backed out.

Right.

> On another note, I believe xulrunner nightly builds should show up on
> Firefox's tbpl trees (m-c, m-a & m-b). Anyone disagrees? IMHO it would help
> catch issues earlier.

I don't disagree, but this is something that should be raised more widely.  I'll bring it up at the engineering meeting tomorrow.
Comment 37 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-11 15:22:09 PDT
Sigh, apparently the Mac XR builds randomly turn red ... and it looks like landing this patch coincided with that.
Comment 38 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-11 15:35:31 PDT
Trying this again ...

http://hg.mozilla.org/mozilla-central/rev/8753de11b181
Comment 39 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-11 21:12:07 PDT
And this was random XR mac bustage.
Comment 40 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-11 21:15:02 PDT
Comment on attachment 537710 [details] [diff] [review]
Iterate over all component manifests when commenting out binary components during omnijar packing

Drivers, we will need to take this on Aurora and Beta so that third parties can build binary addons.
Comment 41 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-12 09:30:46 PDT
Filed Bug 670951 for the intermittent Mac failures.
Comment 42 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-12 10:17:46 PDT
Alex, can you verify that the SDKs you need are all on present on FTP for mozilla-central?
Comment 43 Alex Vincent [:WeirdAl] 2011-07-12 12:31:41 PDT
Confirmed: ftp://ftp.mozilla.org/pub/xulrunner/nightly/latest-mozilla-central/xulrunner-8.0a1.en-US.win32.zip is there for me to pull, if that were the one I wanted.  (We're sticking to Aurora builds.  :-) )

Also, tinderbox shows XULRunner on mozilla-central building that .sdk.zip file.  That would be the one I need from Aurora, and now Beta.
Comment 44 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-12 12:34:36 PDT
Yeah, I'll get it on the branches asap.
Comment 46 Alex Vincent [:WeirdAl] 2011-07-13 09:41:39 PDT
:-D I now have SDK's for FF6 Beta and FF7 Aurora from ftp.mozilla.org - thanks!
Comment 47 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-07-13 09:54:52 PDT
Yay!
Comment 48 :Ehsan Akhgari 2011-08-16 10:11:15 PDT
This seems to have landed on Aurora, so we didn't transplant it to beta during migration:

http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a
Comment 49 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-08-16 10:12:32 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #48)
> This seems to have landed on Aurora, so we didn't transplant it to beta
> during migration:
> 
> http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a

What does that mean?  It came along with the merge, no?
Comment 50 :Ehsan Akhgari 2011-09-12 15:55:35 PDT
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #49)
> (In reply to Ehsan Akhgari [:ehsan] from comment #48)
> > This seems to have landed on Aurora, so we didn't transplant it to beta
> > during migration:
> > 
> > http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a
> 
> What does that mean?  It came along with the merge, no?

No, it was landed separately by you: http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?changeset=e0a0e1800c6a

I just wanted to mention that this change did not make it into Beta.  If you think it should, please nominate it.  Thanks!
Comment 51 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-09-12 17:15:41 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #50)
> (In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #49)
> > (In reply to Ehsan Akhgari [:ehsan] from comment #48)
> > > This seems to have landed on Aurora, so we didn't transplant it to beta
> > > during migration:
> > > 
> > > http://hg.mozilla.org/releases/mozilla-aurora/rev/e0a0e1800c6a
> > 
> > What does that mean?  It came along with the merge, no?
> 
> No, it was landed separately by you:
> http://hg.mozilla.org/releases/mozilla-aurora/
> pushloghtml?changeset=e0a0e1800c6a
> 
> I just wanted to mention that this change did not make it into Beta.  If you
> think it should, please nominate it.  Thanks!

http://hg.mozilla.org/releases/mozilla-beta/rev/e0a0e1800c6a

Sure looks like it's in beta to me ...
Comment 52 :Ehsan Akhgari 2011-09-13 09:54:44 PDT
Hmm, then I don't remember why I commented here during the migration, but there _was_ a reason... :/

Note You need to log in before you can comment on or make changes to this bug.