Last Comment Bug 867259 - Build reliably fails on case-insensitive HFS+ on Mac OS
: Build reliably fails on case-insensitive HFS+ on Mac OS
Status: RESOLVED INCOMPLETE
:
Product: Firefox OS
Classification: Client Software
Component: GonkIntegration (show other bugs)
: unspecified
: x86 Mac OS X
: -- normal (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-04-30 10:38 PDT by Ben Kelly [PTO Aug 26, back Aug 29][:bkelly]
Modified: 2013-09-27 13:37 PDT (History)
5 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Pointer to Github pull request: https://github.com/mozilla-b2g/gonk-patches/pull/1 (370 bytes, text/html)
2013-05-01 15:48 PDT, Ben Kelly [PTO Aug 26, back Aug 29][:bkelly]
no flags Details

Description Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-04-30 10:38:59 PDT
Currently the kernel has a number of files that have the same name, but with different case.  This is problematic on case-insensitive file systems like the default HFS+ on Mac OS.  On these platforms you will see things like the following in ./kernel after a fresh ./config.sh:

> # On branch b2g_autogen_ephemeral_branch
> # Changes not staged for commit:
> #   (use "git add <file>..." to update what will be committed)
> #   (use "git checkout -- <file>..." to discard changes in working directory)
> #
> #       modified:   include/linux/netfilter/xt_CONNMARK.h
> #       modified:   include/linux/netfilter/xt_DSCP.h
> #       modified:   include/linux/netfilter/xt_MARK.h
> #       modified:   include/linux/netfilter/xt_RATEEST.h
> #       modified:   include/linux/netfilter/xt_TCPMSS.h
> #       modified:   include/linux/netfilter_ipv4/ipt_ECN.h
> #       modified:   include/linux/netfilter_ipv4/ipt_TTL.h
> #       modified:   include/linux/netfilter_ipv6/ip6t_HL.h
> #       modified:   net/ipv4/netfilter/ipt_ECN.c
> #       modified:   net/netfilter/xt_DSCP.c
> #       modified:   net/netfilter/xt_HL.c
> #       modified:   net/netfilter/xt_RATEEST.c
> #       modified:   net/netfilter/xt_TCPMSS.c

The "modifications" are only due to the differences within the files with the same name:

  https://github.com/mozilla-b2g/codeaurora_kernel_msm/blob/master/include/linux/netfilter/xt_CONNMARK.h
  https://github.com/mozilla-b2g/codeaurora_kernel_msm/blob/master/include/linux/netfilter/xt_connmark.h

These git differences then cause the ./build.sh to fail with the following error:

> [entering kernel]
> ERROR: You have uncommited changes in kernel
> You may force overwriting these changes
> with |source build/envsetup.sh force|
> 
> ERROR: Patching of kernel/ failed.

The case-sensitivity issue is mentioned in the wiki for the galaxy s2:

  https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Firefox_OS_build_prerequisites#Samsung_Galaxy_S2

> Note: Mac OS X uses a case insensitive filesystem by default, which will prevent you from building Firefox OS down the road (EDITOR'S NOTE: I have never had a problem with this).

I've heard of at least two others running into this recently.  They managed to work around the issue, but its unclear how at the moment.  The envsetup.sh force did not work for me.  Checking git status still shows the files being modified in their kernel.

It seems we might want to modify the wiki to highlight this issue.  Also, perhaps we should add a test to bootstrap-mac.sh to check for case-sensitivity.
Comment 1 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-04-30 11:48:23 PDT
Note, it looks like only the hamachi and leo platforms have kernel patches.  So its likely the other platforms will not trigger this failure.

 https://github.com/mozilla-b2g/gonk-patches

It still seems dubious to build a kernel without the correct set of headers, etc.  It seems like preventing this with a fast-fail in ./config.sh might help avoid future bugs.
Comment 2 Justin Lebar (not reading bugmail) 2013-04-30 12:26:45 PDT
> It seems like preventing this with a fast-fail in ./config.sh might help avoid future 
> bugs.

At the risk of breaking nearly every developer who currently builds B2G on a Mac.

The current situation, where we fast-fail for Leo/Hamachi only, seems a lot better than a situation where nobody can build on their Mac without creating a new partition, don't you think?
Comment 3 Justin Lebar (not reading bugmail) 2013-04-30 12:27:32 PDT
> It seems like preventing this with a fast-fail in ./config.sh might help avoid future 
> bugs.

Oh, maybe you mean that config.sh hamachi would fail on mac on a case-insensitive FS?  That sounds like a good idea to me.
Comment 4 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-04-30 12:31:51 PDT
(In reply to Justin Lebar [:jlebar] from comment #3)
> Oh, maybe you mean that config.sh hamachi would fail on mac on a
> case-insensitive FS?  That sounds like a good idea to me.

I guess it seems to me that compiling against source that has been pseudo-randomly modified due to the case sensitivity of the file system seems dangerous.  So I was suggesting fast-failing for all platforms.

I was also thinking there would be an option to override the check, etc.  Something like "--force" or "--ignore-case-check", etc.  A warning would probably just fly by so fast no one would see it.
Comment 5 Justin Lebar (not reading bugmail) 2013-04-30 12:34:23 PDT
> I guess it seems to me that compiling against source that has been pseudo-randomly 
> modified due to the case sensitivity of the file system seems dangerous.

Indeed it does.  On the other hand we've been doing this for a year now on many systems with no apparent problems.

If we added a warning that you could override, what would that solve?

I'm tempted to say we should fix this bug by unbreaking patcher script for hamachi/leo builds on mac case-insensitive fs.
Comment 6 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-04-30 12:54:00 PDT
(In reply to Justin Lebar [:jlebar] from comment #5)
> Indeed it does.  On the other hand we've been doing this for a year now on
> many systems with no apparent problems.
> 
> If we added a warning that you could override, what would that solve?

Fair enough.  It does seem isolated to netfilter.  I assume we don't use that subsystem?

> I'm tempted to say we should fix this bug by unbreaking patcher script for
> hamachi/leo builds on mac case-insensitive fs.

What were you thinking?  Modifying the check for existing changes to ignore this precise set of modified files?  Or patching out the trouble files in our fork before getting to this point?
Comment 7 Justin Lebar (not reading bugmail) 2013-04-30 13:18:13 PDT
> I assume we don't use that subsystem?

I have no idea.  :)

> What were you thinking?  Modifying the check for existing changes to ignore this precise set of 
> modified files?  Or patching out the trouble files in our fork before getting to this point?

I don't know enough about our patching system to have an opinion.  Either one sounds like an improvement over broken builds to me!
Comment 8 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-04-30 16:59:04 PDT
I'll have to look at it closer, but my current thinking would be to do the following hackery:

1) Check FS for case-sensitivity.  If it is case-sensitive, then we're done.
2) Check kernel config for netfilter. Error out if it is enabled.
3) Check patches for changes to netfilter.  Error out if this is the case.
4) Spam .gitignore with the set of known bad files to prevent error.
5) Proceed with patching.
Comment 9 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-05-01 15:48:36 PDT
Created attachment 744315 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gonk-patches/pull/1

Pointer to Github pull-request
Comment 10 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-05-01 15:51:54 PDT
Note, the submitted patch currently only does steps 1, 4, and 5 from comment 8.  I wanted to get more feedback on the direction before adding a lot of complicated checking.
Comment 11 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-05-06 13:35:58 PDT
I took the liberty of updating the wiki to highlight this issue a bit better for new developers on mac:

https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Firefox_OS_build_prerequisites$compare?to=389305&from=389297
Comment 12 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-05-06 14:15:44 PDT
Note, the original work around provided in attachment 744315 [details] did not work well when switching branches with ./config.sh.  I've since reworked the patch and rebased the pull request to reflect the new approach.
Comment 13 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-05-07 08:34:13 PDT
Comment on attachment 744315 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gonk-patches/pull/1

I closed this pull request because I found the latest patch was not reliable either.  It seems there are a number of corner cases here where git breaks with case insensitive file systems and files only differing in case.
Comment 14 Ben Kelly [PTO Aug 26, back Aug 29][:bkelly] 2013-05-14 07:29:47 PDT
So after trying to work around this issue for a while, I've basically punted and setup a case-sensitive disk image that I mount to work on FFOS.  The instructions are on the wiki, but for completeness here:

> hdiutil create -volname 'firefoxos' -type SPARSE -fs 'Case-sensitive Journaled HFS+' -size 40g ~/firefoxos.sparseimage
> open ~/firefoxos.sparseimage
> cd /Volumes/firefoxos/

This shouldn't be difficult for most people as it doesn't require separate partitions/disks and it really doesn't use any extra space.

Based on that I've come back to the position that we should fast fail when building if we detect case insensitive and spit out these instructions to fix it.
Comment 15 Justin Lebar (not reading bugmail) 2013-05-27 19:20:46 PDT
> Based on that I've come back to the position that we should fast fail when building if we 
> detect case insensitive and spit out these instructions to fix it.

That sounds pretty reasonable to me.

I'll file a new bug.
Comment 16 Justin Lebar (not reading bugmail) 2013-05-27 19:24:25 PDT
Filed bug 876557.
Comment 17 John Ford [:jhford] CET/CEST Berlin Time 2013-08-08 16:31:01 PDT
Anything left to do here?
Comment 18 Justin Lebar (not reading bugmail) 2013-08-08 19:33:40 PDT
I guess not, although the case-sensitive FS emulation layer may have added significantly to build times.  I didn't do a scientific test, though.

Note You need to log in before you can comment on or make changes to this bug.