Last Comment Bug 744008 - Produce B2G builds for desktop operating systems.
: Produce B2G builds for desktop operating systems.
Status: VERIFIED FIXED
[b2g]
:
Product: Release Engineering
Classification: Other
Component: General Automation (show other bugs)
: other
: All All
: P1 normal (vote)
: ---
Assigned To: Ben Hearsum (:bhearsum)
: Chris AtLee [:catlee]
Mentors:
Depends on: 760138 760168 761868 770624 770628 770990
Blocks: 763611
  Show dependency treegraph
 
Reported: 2012-04-10 07:58 PDT by Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)
Modified: 2013-08-12 21:54 PDT (History)
31 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
+


Attachments
desktop build mozconfigs (5.06 KB, patch)
2012-06-01 12:18 PDT, John Ford [:jhford] CET/CEST Berlin Time
cjones.bugs: review+
jhford: checked‑in+
Details | Diff | Splinter Review
move b2g mozconfigs to match up with new platform names (2.89 KB, patch)
2012-06-29 08:35 PDT, Ben Hearsum (:bhearsum)
no flags Details | Diff | Splinter Review
rename mozconfigs; adjust compiler (3.05 KB, patch)
2012-07-03 06:24 PDT, Ben Hearsum (:bhearsum)
gal: review+
bhearsum: checked‑in+
Details | Diff | Splinter Review

Description Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2012-04-10 07:58:32 PDT
Similar to bug 719491 but building for desktop this time, this will let contributors play with B2G/Gaia on Windows/Mac/Linux/... without having to compile B2G on desktop themselves.
Comment 1 Chris Cooper [:coop] [away until Aug 29] 2012-04-12 13:14:07 PDT
Vivien: are there instructions for how to do these builds on desktop, hopefully with configs?
Comment 2 John Ford [:jhford] CET/CEST Berlin Time 2012-04-12 14:06:33 PDT
I did a mac-10.7 build with the following mozconfig:

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-b2g
mk_add_options MOZ_MAKE_FLAGS="-j8"
ac_add_options --enable-application=b2g
ac_add_options --enable-debug-symbols
ac_add_options --enable-profiling
ac_add_options --with-ccache
ac_add_options --enable-marionette
export CXXFLAGS=-DMOZ_ENABLE_JS_DUMP

and when I try to run the resulting b2g binary on the command line, I get a segfault:

~/mozilla/desktop-b2g/obj-b2g/dist $ DYLD_LIBRARY_PATH=lib/ gdb ./bin/b2g
<snip>
Reading symbols for shared libraries . done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000009
nsContainerFrame::SyncFrameViewProperties (aPresContext=0x10966b000, aFrame=0x109690c28, aStyleContext=0x0, aView=0x1, aFlags=0) at nsIView.h:116
116	  { return reinterpret_cast<nsIViewManager*>(mViewManager); }

Is this to be expected or am I doing something wrong?
Comment 3 Chris Jones [:cjones] inactive; ni?/f?/r? if you need me 2012-04-12 19:42:45 PDT
This looks like a null pointer crash.  What version of mozilla-central are you trying?
Comment 4 John Ford [:jhford] CET/CEST Berlin Time 2012-04-12 19:54:26 PDT
901dfde60183, i'll try updating
Comment 5 Chris Jones [:cjones] inactive; ni?/f?/r? if you need me 2012-04-12 19:59:51 PDT
That rev should be fine AFAIK.  You're probably seeing a new gecko bug.  Please file! :)
Comment 6 John Ford [:jhford] CET/CEST Berlin Time 2012-04-12 20:23:25 PDT
(In reply to Chris Jones [:cjones] [:warhammer] from comment #5)
> That rev should be fine AFAIK.  You're probably seeing a new gecko bug. 
> Please file! :)

bug 745070, but I wouldn't be surprised if I filed in the wrong component
Comment 7 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2012-04-13 03:32:57 PDT
(In reply to John Ford [:jhford] from comment #2)
> I did a mac-10.7 build with the following mozconfig:
> 
> mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-b2g
> mk_add_options MOZ_MAKE_FLAGS="-j8"
> ac_add_options --enable-application=b2g
> ac_add_options --enable-debug-symbols
> ac_add_options --enable-profiling
> ac_add_options --with-ccache
> ac_add_options --enable-marionette
> export CXXFLAGS=-DMOZ_ENABLE_JS_DUMP
> 


Let's add those options too:
ac_add_options --enable-tests
ac_add_options --enable-b2g-ril
Comment 8 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2012-04-13 03:33:33 PDT
(In reply to Chris Cooper [:coop] from comment #1)
> Vivien: are there instructions for how to do these builds on desktop,
> hopefully with configs?

The usual configuration people use is the one at: https://wiki.mozilla.org/Gaia/Hacking
Comment 9 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-04-18 14:48:44 PDT
(In reply to Chris Cooper [:coop] from comment #1)
> Vivien: are there instructions for how to do these builds on desktop,
> hopefully with configs?

:coop, bug 732467 discusses some issues with building on OS X; I worked through these on my machine and got a wrking build with the standard options from the wiki, *and* enabling clang as the compiler as the bug suggests:

export CC=clang
export CXX=clang++

Otherwise, I would get a segfault on start.
Comment 10 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-04-19 21:24:35 PDT
*bump*, coop - any feedback on this?
Comment 11 Chris Cooper [:coop] [away until Aug 29] 2012-04-20 06:16:40 PDT
(In reply to Jeff Griffiths from comment #10)
> *bump*, coop - any feedback on this?

jhford seems to be doing the work here, so let's assign it to him.
Comment 12 Chris Jones [:cjones] inactive; ni?/f?/r? if you need me 2012-05-24 18:40:15 PDT
We would love to have these for b2g builds themselves, because xpcshell was recently stripped from the nightly xulrunner package and we need xpcshell to prebuild some DBs.
Comment 13 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-05-25 08:56:57 PDT
cc'ing Chris Lee and Dietrich. Guys, b2g desktop builds are a critical part of the tactics plan we're putting together specifically to supply web developers with the best tools and resources we can to build and test apps for b2g. What is blocking this? Does it need to be a higher priority?
Comment 14 Chris Cooper [:coop] [away until Aug 29] 2012-05-25 12:53:34 PDT
(In reply to Jeff Griffiths from comment #13)
> cc'ing Chris Lee and Dietrich. Guys, b2g desktop builds are a critical part
> of the tactics plan we're putting together specifically to supply web
> developers with the best tools and resources we can to build and test apps
> for b2g. What is blocking this? Does it need to be a higher priority?

jhford no longer works in releng, which might be part of the problem here.

jhford: are you still planning on setting these up since you're on the b2g team now? You didn't hand this bug off when you moved over so I'm unsure.
Comment 15 Chris Lee [:clee] 2012-05-30 10:37:22 PDT
I believe jhford is still working on this as he transitions over to the B2G team.  He should be able to help address the issues we're running into now.  If not, he should ping cjones for help.  Thanks.
Comment 16 John Ford [:jhford] CET/CEST Berlin Time 2012-05-30 13:04:19 PDT
Yes, I am working on this.  I can get patches together that I think would work, but I don't think I have access to test them beyond running ./setup-masters.py -t

I've verified that I can build b2g desktop locally on Mac OS X, so depending on how my Try job goes for the other platforms, we might want a per-platform rollout.
Comment 17 John Ford [:jhford] CET/CEST Berlin Time 2012-05-31 09:21:18 PDT
By submitting a try job that uses a B2G mozconfig in place of a Firefox one, I've been able to test whether B2G builds on the Releng build hosts.  These builds are built with the same toolchain that desktop Firefox is built with, so should have the same runtime dependencies and compatibility.

32-bit Linux:
Works

64-bit Linux:
Works

64-bit Mac OS X 10.7:
Builds, "make buildsymbols" doesn't.  I am going to look into the mozconfig.

32-bit Windows:
Fails to build while building b2g.exe because of redit.exe not being built.  Filed 760138 to track fixing this.

I am going to file a bug for the buildbot-configs changes needed, the mozconfigs to be checked into mozilla-central/b2g will be attached to this bug.
Comment 18 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-05-31 10:36:03 PDT
(In reply to John Ford [:jhford] from comment #17)
...
> 64-bit Mac OS X 10.7:
> Builds, "make buildsymbols" doesn't.  I am going to look into the mozconfig.

Which cc on 10.7 are you using? I could only get my builds to work with clang from XCode 4.2 - but I assume that is the standard Firefox toolchain?
Comment 19 Hernán Rodriguez Colmeiro (:peregrino) 2012-05-31 10:40:11 PDT
(In reply to Jeff Griffiths from comment #18)
> (In reply to John Ford [:jhford] from comment #17)
> ...
> > 64-bit Mac OS X 10.7:
> > Builds, "make buildsymbols" doesn't.  I am going to look into the mozconfig.
> 
> Which cc on 10.7 are you using? I could only get my builds to work with
> clang from XCode 4.2 - but I assume that is the standard Firefox toolchain?

I also had some issues on 10.6 regarding the packaging, so I had to patch some files. I can attach the patch here if it's needed.
Comment 20 John Ford [:jhford] CET/CEST Berlin Time 2012-05-31 10:54:48 PDT
This is on the buildbot build machines.  Those machines are using Xcode 4.1 and have real gcc-4.2 available.  They build just fine, it's that the buildsymbols target isn't included in the objdir Makefile.
Comment 21 John Ford [:jhford] CET/CEST Berlin Time 2012-06-01 12:18:09 PDT
Created attachment 629290 [details] [diff] [review]
desktop build mozconfigs

These mozconfigs build on the try server when put in as the browser/configs/mozconfigs/ equivalents.  The Linux builds both fail l10n checks, which will be disabled in the buildbot patch.  The mac patch was tested using a couple extra lines to force the objdir to be "obj-firefox/i386" to work around a couple issues with how buildbot picks the directory to run objdir makefile targets.  Windows dies trying to build b2g.exe, but that's while trying to run redit.exe, which isn't being built.  Bug 760138 tracks fixing the windows builds.
Comment 22 John Ford [:jhford] CET/CEST Berlin Time 2012-06-04 10:36:56 PDT
Comment on attachment 629290 [details] [diff] [review]
desktop build mozconfigs

https://hg.mozilla.org/mozilla-central/rev/5966ab6ebcca
Comment 23 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2012-06-04 11:14:14 PDT
I wonder if it make sense to add to the build:
 ac_add_options --enable-b2g-bt
 ac_add_options --enable-b2g-ril
Comment 24 John Ford [:jhford] CET/CEST Berlin Time 2012-06-05 11:47:02 PDT
(In reply to Vivien Nicolas (:vingtetun) from comment #23)
> I wonder if it make sense to add to the build:
>  ac_add_options --enable-b2g-bt
>  ac_add_options --enable-b2g-ril

Sure!  Is it OK if we wait until these builds are green in production to do this?
Comment 25 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2012-06-06 03:28:44 PDT
(In reply to John Ford [:jhford] from comment #24)
> (In reply to Vivien Nicolas (:vingtetun) from comment #23)
> > I wonder if it make sense to add to the build:
> >  ac_add_options --enable-b2g-bt
> >  ac_add_options --enable-b2g-ril
> 
> Sure!  Is it OK if we wait until these builds are green in production to do
> this?

Sounds good to me :)
Comment 26 Jason Smith [:jsmith] 2012-06-11 18:11:19 PDT
Nominating for k9o and basecamp - this is badly needed to be able to crowd source testing for b2g for contributors who want to help (we need all the help we can get for QA testing to get more testing).
Comment 27 Tony Chung [:tchung] 2012-06-11 22:37:19 PDT
Can we include these desktop builds all in the same location where the daily QA builds live?   See bug 763611 and bug 761868.
Comment 28 Hernán Rodriguez Colmeiro (:peregrino) 2012-06-12 06:38:18 PDT
(In reply to Tony Chung [:tchung] from comment #27)
> Can we include these desktop builds all in the same location where the daily
> QA builds live?   See bug 763611 and bug 761868.

As far as I know, those builds are not available for distribution outside mozilla due to licenses stuff on the binaries used to produce them. Desktop B2G is not tied to those binaries and can be freely distributed, so I would like this builds to be on a public reachable location.
Comment 29 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-06-12 10:28:11 PDT
These builds should end up on the public FTP, ASAP.
Comment 30 John Ford [:jhford] CET/CEST Berlin Time 2012-06-12 10:43:33 PDT
(In reply to Jeff Griffiths from comment #29)
> These builds should end up on the public FTP, ASAP.

Jeff, this is blocked on bug 760168 which is adding the b2g desktop platforms to the releng configurations.
Comment 31 Tony Chung [:tchung] 2012-06-12 11:02:54 PDT
(In reply to Hernán Rodriguez Colmeiro (:peregrino) from comment #28)
> (In reply to Tony Chung [:tchung] from comment #27)
> > Can we include these desktop builds all in the same location where the daily
> > QA builds live?   See bug 763611 and bug 761868.
> 
> As far as I know, those builds are not available for distribution outside
> mozilla due to licenses stuff on the binaries used to produce them. Desktop
> B2G is not tied to those binaries and can be freely distributed, so I would
> like this builds to be on a public reachable location.

Correct, mozilla cannot distribute the device builds.  But mozilla can distribute the manifest files that go with the device builds.  And emulator builds are okay.

I'm just proposing if we can do something like https://releases.mozilla.com/b2g/public, where the public FTP directory folders contains the following:
- Daily Manifest file for device builds
- Daily emulator builds (x86 and ARM)
- Daily b2g Desktop builds
Comment 32 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-06-12 11:18:00 PDT
(In reply to Tony Chung [:tchung] from comment #31)
...
> I'm just proposing if we can do something like
> https://releases.mozilla.com/b2g/public, where the public FTP directory
> folders contains the following:
> - Daily Manifest file for device builds
> - Daily emulator builds (x86 and ARM)
> - Daily b2g Desktop builds

That would indeed be splendid.
Comment 33 Hernán Rodriguez Colmeiro (:peregrino) 2012-06-12 12:06:11 PDT
(In reply to Tony Chung [:tchung] from comment #31)
> Correct, mozilla cannot distribute the device builds.  But mozilla can
> distribute the manifest files that go with the device builds.  And emulator
> builds are okay.
> 
> I'm just proposing if we can do something like
> https://releases.mozilla.com/b2g/public, where the public FTP directory
> folders contains the following:
> - Daily Manifest file for device builds
> - Daily emulator builds (x86 and ARM)
> - Daily b2g Desktop builds

If that can be done, would be great to have. Will those B2G Desktop builds update like nightly does or that is an entire new issue that should be filed?
Comment 34 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-06-20 14:11:47 PDT
*bump* :jhford, what is the status of this? We would like to have publicly available desktop builds for people to be able test apps with at various hack days we are planning in Brazil and around the world, starting in early July. Is this possible?
Comment 35 Tony Chung [:tchung] 2012-06-20 14:14:39 PDT
(In reply to Jeff Griffiths from comment #34)
> *bump* :jhford, what is the status of this? We would like to have publicly
> available desktop builds for people to be able test apps with at various
> hack days we are planning in Brazil and around the world, starting in early
> July. Is this possible?

Jeff, you should comment in the etherpad below that releng is tracking for build work. See #9 in particular.   As far as i can read, there's not a compelling enough reason to proceed with desktop buids.

https://etherpad.mozilla.org/b2g-builds
Comment 36 John O'Duinn [:joduinn] (please use "needinfo?" flag) 2012-06-28 09:29:39 PDT
per irc, jhford is focusing on bug#769315 instead so reassigning.
Comment 37 Ben Hearsum (:bhearsum) 2012-06-29 04:26:57 PDT
When testing the RelEng side of things over in bug 760168 I got the following error from the Windows build:
e:/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox/config/nsinstall.exe -D ../../dist/bin/chrome/icons/default
cp /e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/b2g/app/b2g.ico ../../dist/bin/chrome/icons/default/b2g.ico
e:/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox/dist/bin/redit.exe ../../dist/bin/b2g.exe /e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/b2g/app/b2g.ico
make[6]: Leaving directory `/e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox/b2g/app'
make[5]: Leaving directory `/e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox/b2g'
make[4]: Leaving directory `/e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox'
make[3]: Leaving directory `/e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox'
make[2]: Leaving directory `/e/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox'
make[1]: Leaving directory `/e/builds/moz2_slave/m-cen-w32-gecko-ntly/build'
make[6]: e:/builds/moz2_slave/m-cen-w32-gecko-ntly/build/obj-firefox/dist/bin/redit.exe: Command not found

Anyone know how to fix this?
Comment 38 Ben Hearsum (:bhearsum) 2012-06-29 08:35:15 PDT
Created attachment 637916 [details] [diff] [review]
move b2g mozconfigs to match up with new platform names

And this patch removes the old "linux32" mozconfig, which was actually for the gb_armv7a_gecko build.
Comment 39 Ben Hearsum (:bhearsum) 2012-07-03 06:24:44 PDT
Created attachment 638687 [details] [diff] [review]
rename mozconfigs; adjust compiler

Same patch as before, except I've removed the $topsrcdir/build/unix/mozconfig.linux inheritance, because it sets a compiler that's not available on the slaves we'll be using to build the Linux desktop builds.

Gal, for background: The original mozconfigs were created with different platform names that we ended up using. This patch's purpose is to rename them to make their names consistent with the ones we're using in bug 760168.
Comment 40 Ben Hearsum (:bhearsum) 2012-07-03 06:34:55 PDT
Comment on attachment 638687 [details] [diff] [review]
rename mozconfigs; adjust compiler

Landed: http://hg.mozilla.org/mozilla-central/rev/f38d6df93cad
Comment 41 Ben Hearsum (:bhearsum) 2012-07-03 10:38:40 PDT
It seems to me that this *blocks* bug 763611, not depends on it.
Comment 42 Ben Hearsum (:bhearsum) 2012-07-10 05:14:06 PDT
bugs 770625 and 771095 don't block this because they don't prevent us from producing builds. The former is a nice-to-have that may never get fixed and the latter is a build config issue that needs to be fixed before the Windows builds will run. Once bug 770990 is fixed we'll be creating and uploading builds for Linux, Mac and Windows and IMO be done here.
Comment 43 Ben Hearsum (:bhearsum) 2012-07-13 10:34:14 PDT
Comment on attachment 638687 [details] [diff] [review]
rename mozconfigs; adjust compiler

This was checked in awhile ago.
Comment 44 Ben Hearsum (:bhearsum) 2012-07-13 10:34:38 PDT
Linux, Mac and Windows builds are running, uploading, and working now.
Comment 46 Tony Chung [:tchung] 2012-07-24 18:31:29 PDT
Verified fixes.

Note You need to log in before you can comment on or make changes to this bug.