Use libdmg-hfsplus for generating DMG images when packaging Mac build on Linux

RESOLVED FIXED in Firefox 43

Status

()

Core
Build Config
RESOLVED FIXED
4 years ago
2 years ago

People

(Reporter: joey, Assigned: ted)

Tracking

unspecified
mozilla43
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox43 fixed)

Details

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(5 attachments)

Comment hidden (empty)
(Reporter)

Updated

4 years ago
Assignee: nobody → joey
(Reporter)

Comment 1

4 years ago
maloader not currently an option for running hdiutil directly.  The project owner mentioned direct access to apple framework libraries are not supported and he did not have plans to add it.

The darling project {maloader v2} may not be stable enough yet for production use.  The project has several notices posted on the site that the code base is developer quality and no ETA for adding specific features or supporting programs.

Trying out the hfsplus filesystem on linux.  Storing a checksum for the disk image will be one of the problems to figure out.
Note that dmg don't *have* to be hfs. IIRC, iso9660 is also supported.
(Reporter)

Comment 3

4 years ago
(In reply to Mike Hommey [:glandium] from comment #2)
> Note that dmg don't *have* to be hfs. IIRC, iso9660 is also supported.

Will add this as an option to look at but we should probably try and keep the existing storage format if possible so the cross builds so they do not stray very far from the native generated image.
(Reporter)

Comment 4

4 years ago
hfs/hdiutil related problems:
  - hdiutil verify will validate a disk image against a checksum stored in the image.
    - a checksum would not be needed for regular builds, only for nightly/release(d) images.
  - File mounted for hfs I/O is a fixed size and will need to be pre-created:
    - hdiutil on mac supports a resize argument if the size of wasted space will be an issue.

A disk image generated on linux can be mounted and interacted on a mac laptop.
May be coincidence but launchd caused a cpu panic around the time an image was mounted/unmounted several times.  Has only happened once but need to keep an eye on it.
(Assignee)

Updated

4 years ago
Duplicate of this bug: 921494
(Assignee)

Comment 6

4 years ago
Duped my bug over here because you have discussion here. My initial comment over there:
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #0)
> We might want to do OS X builds on Linux. One thing that's currently hard is
> producing DMG files on non-Mac.
> 
> A cursory Google search indicates that people have had success with this
> library:
> https://github.com/hamstergene/libdmg-hfsplus
> per http://stackoverflow.com/a/17735033/69326
(Assignee)

Updated

4 years ago
Blocks: 921040
Blocks: 927061
No longer blocks: 921040
(Assignee)

Updated

4 years ago
Blocks: 921040
(Reporter)

Comment 7

4 years ago
Notes from test of creating a dmg image.  Posted on the cross bug because mike's download was used as a source.

https://bugzilla.mozilla.org/show_bug.cgi?id=921040#c106
we can roll out compile without dmg support, so leaving this bug as only blocking our deployment strategy (bug 927061)
No longer blocks: 921040
Status update -- while functional (linux DMG usable for running app on osx), there are two known current issues:

- nightly & release builds use a checksum option for the DMG supported by hdiutil. We don't know yet if it's possible to generate the same checksum outside of hdiutil.

- linux DMG are about 2x the size of current release builds. This appears to be due to compression happening somewhere -- uncompressed app sizes are comparable.

Investigation on both is continuing.
(Assignee)

Comment 10

4 years ago
I'd be interested to see someone try the tools I linked in comment 6, as others have reported success with them on Stackoverflow.
(In reply to Joey Armstrong [:joey] from comment #4)
> hfs/hdiutil related problems:
>   - hdiutil verify will validate a disk image against a checksum stored in
> the image.
>     - a checksum would not be needed for regular builds, only for
> nightly/release(d) images.

I think I mentioned this last week, but I'm not sure I agree with this. Consistency between dep and nightly/release builds is pretty important, especially on beta (where we only have dep and "release"). I think we should aim to keep packaging the same for all of these. We talked about the idea of a packaging service (similar to the signing servers) that would run on Mac machines. If we have to build something like that for nightly/release builds, we may as well use it for dep builds too.

Comment 12

4 years ago
(In reply to Hal Wine [:hwine] (use needinfo) from comment #9)
> - linux DMG are about 2x the size of current release builds. This appears to
> be due to compression happening somewhere -- uncompressed app sizes are
> comparable.

Are you currently just loop-mounting an HFS+ image? That produces raw images (CRawDiskImage), that don't really use the .dmg format (UDIF variants). The dmgs distributed by Mozilla now use the UDBZ (UDIF bzip2-compressed) type.

I don't think there will be any realistic way to add compression or checksums to raw images. You'll need to convert it somehow to UDIF, with something like libdmg-hfsplus or a newly-written utility.

Comment 13

4 years ago
In addition to libdmg-hfsplus, 7-zip currently contains some code related to UDIF dmg files. I don't know if it can create them, perhaps it's read-only, but it's interesting reading.
(Assignee)

Comment 14

4 years ago
I didn't have much success with libdmg-hfsplus. It makes a DMG, but trying to mount it on my 10.8 Mac gives an "Invalid checksum" error. I can weasel around this by opening the DMG with Disk Utility, clicking "Repair", and then mounting it from there, but clearly libdmg-hfsplus is not doing the right thing.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #14)
> I didn't have much success with libdmg-hfsplus. It makes a DMG, but trying
> to mount it on my 10.8 Mac gives an "Invalid checksum" error. I can weasel
> around this by opening the DMG with Disk Utility, clicking "Repair", and
> then mounting it from there, but clearly libdmg-hfsplus is not doing the
> right thing.

OOC, did you try some sort of binary diff between the before and after to see how trivial the repairing is?  Maybe it's not hard to fix libdmg-hfsplus...
Flags: needinfo?(ted)

Comment 16

4 years ago
I don't know exactly what y'all are attempting to package up in a dmg. But here's what I did to successfully create a dmg with a random dir on my drive. I'm running OS X, so I guess it's possible some step isn't working on Linux, I'll test that out later.

git clone https://github.com/erwint/libdmg-hfsplus
cd libdmg-hfsplus
cmake .
cd dmg
make

mkisofs -D -V "test" -no-pad -R -apple -o test.iso $(MY_DIR)
./dmg dmg test.iso test.dmg
hdiutil attach test.dmg # It mounts!
(Assignee)

Comment 17

4 years ago
(In reply to Nathan Froyd (:froydnj) from comment #15)
> OOC, did you try some sort of binary diff between the before and after to
> see how trivial the repairing is?  Maybe it's not hard to fix
> libdmg-hfsplus...

It doesn't seem to actually change the on-disk image. Maybe it's just an easy way to confuse it into ignoring the checksum.

(In reply to Dave Vasilevsky from comment #16)
> I don't know exactly what y'all are attempting to package up in a dmg. But
> here's what I did to successfully create a dmg with a random dir on my
> drive. I'm running OS X, so I guess it's possible some step isn't working on
> Linux, I'll test that out later.
> 
> git clone https://github.com/erwint/libdmg-hfsplus

Hm, I was apparently using a different fork: https://github.com/hamstergene/libdmg-hfsplus

I've been doing:
cd $objdir/dist
mkdir /tmp/bundle
rsync -rL NightlyDebug.app /tmp/bundle
genisoimage -D -V "Firefox test" -no-pad -r -apple -o firefox-uncompressed.dmg /tmp/bundle/

cd /build/libdmg-hfsplus/dmg
./dmg dmg /build/obj-firefox-darwin/dist/firefox-uncompressed.dmg /build/obj-firefox-darwin/dist/firefox.dmg

...switching to that other fork seems to fix it. Huh!
Flags: needinfo?(ted)
(Assignee)

Comment 18

4 years ago
Note that the DMG produced with this tool is UDZO format, which is zlib-compressed, where our current DMGs are UDBZ, bzip2-compressed.

Comment 19

4 years ago
It's not at all hard to add bzip2 support. Here's a first stab, it seems to work so far: https://github.com/vasi/libdmg-hfsplus/tree/bzip2 . Ideally there would be a command-line flag or something, but the option parsing is terrible so I'm holding off on that for now.

Comment 20

4 years ago
Hmm, and I just realized that we're only compressing 100k at a time, so it makes little sense to use level-9 compression for bzip2. I think bzip2 blocks store their compression level in the header, I should try to inspect a UDBZ created by Apple tools and figure out what they use.

Comment 21

4 years ago
Well, that's unexpected:

$ hdiutil create -srcfolder ~/SDKs/MacOSX10.7.sdk -format UDBZ bz.dmg
$ strings bz.dmg  | perl -ne 'print "$1\n" if /BZh(\d)/' | sort | uniq -c
   1 0
 187 1
  19 2
 708 3

Why would we get a mix of 1 and 3? Are Apple's tools choosing dynamically? I guess we should just use level 3 by default.
(Assignee)

Comment 22

4 years ago
Dave: thanks for that! I think your questions are falling on deaf ears here, you clearly know way more about the DMG format than any of us. :)

Comment 23

4 years ago
So here's what I think we still need:

1. Test that our UDBZ images can be created on Linux and opened on OS X
2. Test that Firefox from such an image can actually be run on OS X, ie: it hasn't lost anything important in the conversion.
3. Set the positions of the file icons in the DMG root folder. I think we can just copy the .DS_Store.
4. Set a background image for the DMG root folder. I think we can just copy a .background file.
5. Include a link to the Applications folder, I think a symlink will work.
6. Set a volume icon for the disk image. This means copying a .VolumeIcon.icns file, but ALSO setting the 'Has Custom Icon' attribute on the root directory. I have no idea if we can do this from Linux.
7. Maybe there are issues with other character encodings? We should probably test.

Anything else?
(Assignee)

Comment 24

4 years ago
I believe 3-6 are already covered by our packaging process, we pass arguments to a script called pkg-dmg:
http://mxr.mozilla.org/mozilla-central/source/build/package/mac_osx/pkg-dmg
http://hg.mozilla.org/mozilla-central/annotate/ae2c044c6418/toolkit/mozapps/installer/packager.mk#l446

It's probably mostly a matter of teaching pkg-dmg (unfortunately a Perl script) to invoke the dmg program instead of hdiutil to create the actual DMG.

Comment 25

4 years ago
If you look at pkg-dmg, it calls 'SetFile -a C' on the root directory to set the "Has Custom Icon" Finder flag. That's clearly not going to work on Linux, right?

I think Linux does support setting Finder flags using an extended attribute, see: http://lxr.free-electrons.com/source/fs/hfsplus/xattr.c#L172 . I think we can do that in Perl with some bit twiddling.
(Assignee)

Comment 26

4 years ago
Not sure how we'd do that in this process though, since that's being called on the host filesystem (which won't be HFS+ on Linux), right? Is that something we could hack into libdmg? I assume it's creating a HFS+ filesystem internally, so maybe we can coerce it to set that attribute?

Comment 27

4 years ago
libdmg doesn't actually create an HFS+ filesystem, it works only at the lower level of blocks and partitions. We would have to either use genisoimage options to add an icon, or give libdmg as input an HFS+ volume.
(Assignee)

Comment 28

4 years ago
Okay, hopefully one of those is feasible.
(Assignee)

Comment 29

4 years ago
Resummarizing this bug, I think we have a winner, we just need to make it work in the build system.
Summary: Check on linux options for generating DMG images → Use libdmg-hfsplus for generating DMG images when packaging Mac build on Linux
(Reporter)

Updated

4 years ago
Assignee: joey → nobody

Comment 30

4 years ago
Is this still being worked on? I can look into the remaining issues further if it would be useful.
(Assignee)

Comment 31

4 years ago
Yes, it is, we just haven't been pushing very hard yet, and packaging is the last step in the whole process. If you could work out the remaining issues that would be fantastic. Ideally we'd be able to just run the "pkg-dmg" script on a Linux host and have it work properly.

Comment 32

4 years ago
Hi guys. I am trying to use https://github.com/vasi/libdmg-hfsplus/ to bundle DMGs from Linux for Tor Browser. That repo does properly create a UDBZ compressed dmg from a genisofs image that opens on OSX 10.6, but I am running into a couple issues though:

1. The app icon seems to be positioned wrong if I use the default dsstore as .DS_Store. It also doesn't seem to be using our background image file (and is instead sourcing /Volumes/Firefox/.background/background.png). Do I need to edit .DS_Store somehow? The dsstore file included in the branding directory is a binary file. How can I generate/edit that? A simple 'sed -i -e "s/Firefox/TorBrowserBundle_en-US/g" .DS_Store' seems to break the background and layout entirely. 

2. Does anyone know a way to get the symlink to be ~/Desktop instead of /Applications? We'd prefer to have the user place TBB into their desktop instead of a system directory, but if this is impossible, perhaps we can just add some text (but how to localize it?), or omit the link entirely. I am not sure which is better from a usability standpoint.
(Assignee)

Comment 33

4 years ago
(In reply to Mike Perry from comment #32)
> 2. Does anyone know a way to get the symlink to be ~/Desktop instead of
> /Applications? We'd prefer to have the user place TBB into their desktop
> instead of a system directory, but if this is impossible, perhaps we can
> just add some text (but how to localize it?), or omit the link entirely. I
> am not sure which is better from a usability standpoint.

You can just locally patch this line here:
http://hg.mozilla.org/mozilla-central/annotate/d9c58306bfbc/browser/installer/Makefile.in#l95

I'm not sure if it'd be worthwhile to have functionality to set that in our codebase.

Comment 34

4 years ago
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #33)
> (In reply to Mike Perry from comment #32)
> > 2. Does anyone know a way to get the symlink to be ~/Desktop instead of
> > /Applications? We'd prefer to have the user place TBB into their desktop
> > instead of a system directory, but if this is impossible, perhaps we can
> > just add some text (but how to localize it?), or omit the link entirely. I
> > am not sure which is better from a usability standpoint.
> 
> You can just locally patch this line here:
> http://hg.mozilla.org/mozilla-central/annotate/d9c58306bfbc/browser/
> installer/Makefile.in#l95
> 
> I'm not sure if it'd be worthwhile to have functionality to set that in our
> codebase.

The problem is that I don't seem to have the ability to create a ln -s symlink from " " to ~/Desktop. The ~ character is not expanded when following the symlink...
(Assignee)

Comment 35

4 years ago
Ah, that makes sense. I don't know any way to make that work, I suspect there's not anything smart about this, it's just a symlink acting the way symlinks do.

Comment 36

4 years ago
I thought we had established that this would likely work best with a real HFS+ filesystem created on Linux, rather than with an ISO? I have a user report of this working pretty well, complete with a volume icon.

Comment 37

4 years ago
(In reply to Dave Vasilevsky from comment #36)
> I thought we had established that this would likely work best with a real
> HFS+ filesystem created on Linux, rather than with an ISO? I have a user
> report of this working pretty well, complete with a volume icon.

Well, the main problem I ran into is that there is no way to create compressed (especially UDBZ) dmgs using any tools I could find, save for https://github.com/vasi/libdmg-hfsplus/. However, perhaps we can use the dmg tool from that repo to turn an HFS+ filesystem into a UDBZ DMG, instead of starting with genisoimage. Though the autosizing of the uncompressed image seems a bit trickier with the Linux mkfs.hfsplus tool than with genisoimage. Any suggestions there?

Comment 38

4 years ago
(In reply to Mike Perry from comment #37)
> However, perhaps we can use the dmg
> tool from that repo to turn an HFS+ filesystem into a UDBZ DMG, instead of
> starting with genisoimage.

Yes, that's the idea. Just run the dmg tool on the HFS+ filesystem.

> Though the autosizing of the uncompressed image
> seems a bit trickier with the Linux mkfs.hfsplus tool than with genisoimage.
> Any suggestions there?

I think it should be safe to create a filesystem much bigger in size than is necessary. The loop-mounted filesystem image can be sparse, so it won't take up much space on the build machine. And libdmg should be able to efficiently handle large amounts of unused space in the filesystem, so I don't think it will make the compressed DMG larger.

Comment 39

4 years ago
I suspect that I am the user Dave referred to in comment 36 above. I never got a working DMG by trying to create an HFS+ filesystem. What *is* working for my project is:

  genisoimage -V volname -D -R -apple -no-pad -o uncompressed.dmg indir
  dmg dmg uncompressed.dmg compressed.dmgq

Comment 40

4 years ago
(In reply to Joel Uckelman from comment #39)
> I suspect that I am the user Dave referred to in comment 36 above.

Surprisingly, there were actually multiple users who emailed me directly about this. Weird!

The user I'm referring to created a template HFS+ filesystem on OS X, complete with volume icon and background image. Then on Linux, they duplicated the template, and copied their app into place on the duplicate.

> What *is* working for my project is:
> 
>   genisoimage -V volname -D -R -apple -no-pad -o uncompressed.dmg indir
>   dmg dmg uncompressed.dmg compressed.dmgq

Were you able to get the volume icon working this way? My impression was that this strategy can't give us a volume icon, at least not without a patch to genisoimage.

Comment 41

4 years ago
(In reply to Dave Vasilevsky from comment #40)
> (In reply to Joel Uckelman from comment #39)
> > I suspect that I am the user Dave referred to in comment 36 above.
> 
> Surprisingly, there were actually multiple users who emailed me directly
> about this. Weird!

I am surprised. :)
 
> The user I'm referring to created a template HFS+ filesystem on OS X,
> complete with volume icon and background image. Then on Linux, they
> duplicated the template, and copied their app into place on the duplicate.
> 
> > What *is* working for my project is:
> > 
> >   genisoimage -V volname -D -R -apple -no-pad -o uncompressed.dmg indir
> >   dmg dmg uncompressed.dmg compressed.dmgq
> 
> Were you able to get the volume icon working this way? My impression was
> that this strategy can't give us a volume icon, at least not without a patch
> to genisoimage.

What do you mean by "volume icon"? I'm not a Mac user myself---I'm only stuck creating software for Macs sometimes.

Comment 42

4 years ago
(In reply to Joel Uckelman from comment #41)
> What do you mean by "volume icon"? I'm not a Mac user myself---I'm only
> stuck creating software for Macs sometimes.

When a user opens a .dmg file, it mounts as a virtual disk. That disk appears on the user's desktop, and by default it will have a generic icon, like the one in the bottom right of [1]. We would rather use a custom image that looks Firefox-related, here's one of the past versions [2].

[1] http://imss-website-storage.cloud.caltech.edu.s3-us-west-1.amazonaws.com/legacy/mac-manual-install-1.png
[2] http://people.mozilla.com/~faaborg/files/20081216-platformIcons/diskImage-256.png

Comment 43

4 years ago
(In reply to Dave Vasilevsky from comment #42)
> (In reply to Joel Uckelman from comment #41)
> > What do you mean by "volume icon"? I'm not a Mac user myself---I'm only
> > stuck creating software for Macs sometimes.
> 
> When a user opens a .dmg file, it mounts as a virtual disk. That disk
> appears on the user's desktop, and by default it will have a generic icon,
> like the one in the bottom right of [1]. We would rather use a custom image
> that looks Firefox-related, here's one of the past versions [2].

Thanks for the explanation. We didn't attempt to set a volume icon. The man page for the version of genisomage I have (1.1.11) has a section called "HFS CUSTOM VOLUME/FOLDER ICONS" which seems to be saying that you can make an HFS floppy (!) with the volume icon you want on a Mac and then use genisomage to extract the data from it and insert that back into the image you're creating.

Comment 44

4 years ago
FWIW, it appears that libdmg-hfsplus produces compressed DMGs in a non-deterministic, non-reproducible way. Two successive invocations of the 'dmg' utility built from either 'master' or the bzip2 branch from https://github.com/vasi/libdmg-hfsplus/ will produce different output DMGs from the same uncompressed input image. Using libfaketime to normalize the clock does not seem to solve this issue. The differences are limited to two blocks of bytes, and are not extensive.

I have not yet dug deeply into this, but I grepped through the code, and found that it actually contains calls to rand() for some fields in the headers. Adding a call to srand(4) at the top of main in dmg.c did not seem to help, either, so I'm still not entirely sure what the entropy source is for these random bytes at the movement. Unfortunately, I need to put this down for a few days.

Figured I'd mention this here in case someone who is more familiar with the libdmg compression utility will instantly be able to say, "Oh, that's because they frob some of the wuzzles in the DMG during compression. Here's where they do that", and then we can go from there to make that bit reproducible instead of completely random.

This reproducibility property is important for us at Tor because we have created a build process for Linux, Mac, and Windows that allows anyone to compile identical binaries to the ones we distribute using our source code. We do this to ensure build integrity:
https://blog.torproject.org/blog/deterministic-builds-part-one-cyberwar-and-global-compromise
https://blog.torproject.org/blog/deterministic-builds-part-two-technical-details

We are switching to DMG packaging for MacOS, and are able to produce identical uncompressed DMGs reliably, but this dmg compression utility is blocking the final step.

Comment 45

4 years ago
Ok, I found the culprit. It turns out a UDIF struct was being allocated on the stack without being memset, and some of its fields+padding contained uninitialized random memory. I will attach a patch against libdmg-hfsplus here.

So no wuzzles were being frobbed after all. That's good, at least.

Comment 46

4 years ago
Created attachment 8387355 [details] [diff] [review]
Patch against libdmg-hfsplus to ensure reproducible DMG compression

Comment 47

4 years ago
I suspected it was uninitialized memory, sorry I didn't get a chance to check. I'll update my repo soonish, gotta get a reasonable development environment up again.
(Assignee)

Updated

3 years ago
Blocks: 921040
(Assignee)

Updated

3 years ago
Assignee: nobody → ted
(Assignee)

Updated

3 years ago
Depends on: 1190522
(Assignee)

Comment 48

2 years ago
I wired this all up into packaging, try push here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d4552597b453
(Assignee)

Comment 49

2 years ago
This seems to have worked:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=53ddcffbc033

Unfortunately in the meantime bug 1197293 broke TaskCluster builds, so the whole build was not successful.
(Assignee)

Updated

2 years ago
Blocks: 1197325
(Assignee)

Comment 50

2 years ago
Created attachment 8651201 [details]
MozReview Request: bug 935237 - use libdmg-hfsplus to create DMG files during packaging on Linux. r?gps

bug 935237 - use libdmg-hfsplus to create DMG files during packaging on Linux. r?gps
Attachment #8651201 - Flags: review?(gps)
(Assignee)

Comment 51

2 years ago
Created attachment 8651202 [details]
MozReview Request: bug 935237 - add genisoimage+dmg binaries to tooltool manifest, use them in cross-mac mozconfig. r?gps

bug 935237 - add genisoimage+dmg binaries to tooltool manifest, use them in cross-mac mozconfig. r?gps
Attachment #8651202 - Flags: review?(gps)
(Assignee)

Comment 52

2 years ago
Created attachment 8651203 [details]
MozReview Request: bug 1197154 - Hook up cross-mac taskcluster builds on try. r?dustin

bug 1197154 - Hook up cross-mac taskcluster builds on try. r?dustin

Also tweak some things in the task definition, and rename the mozharness
config file to make mozharness happy.
Attachment #8651203 - Flags: review?(dustin)
(Assignee)

Comment 53

2 years ago
Created attachment 8651204 [details]
MozReview Request: bug 1197248 - Don't try to --enable-rust on cross-mac builds yet. r?gps

bug 1197248 - Don't try to --enable-rust on cross-mac builds yet. r?gps
Attachment #8651204 - Flags: review?(gps)
Comment on attachment 8651203 [details]
MozReview Request: bug 1197154 - Hook up cross-mac taskcluster builds on try. r?dustin

https://reviewboard.mozilla.org/r/16869/#review15013

looks good
Attachment #8651203 - Flags: review?(dustin) → review+
Comment on attachment 8651201 [details]
MozReview Request: bug 935237 - use libdmg-hfsplus to create DMG files during packaging on Linux. r?gps

https://reviewboard.mozilla.org/r/16865/#review15183

::: python/mozbuild/mozpack/dmg.py:40
(Diff revision 1)
> +        #TODO: bug 1197325 - figure out how to support this on Linux

Perhaps this should raise an exception until this is implemented?
Attachment #8651201 - Flags: review?(gps)

Updated

2 years ago
Attachment #8651201 - Flags: review+
Comment on attachment 8651201 [details]
MozReview Request: bug 935237 - use libdmg-hfsplus to create DMG files during packaging on Linux. r?gps

https://reviewboard.mozilla.org/r/16865/#review15185

Ship It!
Comment on attachment 8651202 [details]
MozReview Request: bug 935237 - add genisoimage+dmg binaries to tooltool manifest, use them in cross-mac mozconfig. r?gps

https://reviewboard.mozilla.org/r/16867/#review15187

::: build/macosx/cross-mozconfig.common:38
(Diff revision 1)
> +export GENISOIMAGE=$topsrcdir/genisoimage/genisoimage
> +export DMG_TOOL=$topsrcdir/dmg/dmg

Is there not a better place to put these? I'm not a huge fan of littering topsrcdir with untracked files.
Attachment #8651202 - Flags: review?(gps) → review+

Updated

2 years ago
Attachment #8651204 - Flags: review?(gps) → review+
Comment on attachment 8651204 [details]
MozReview Request: bug 1197248 - Don't try to --enable-rust on cross-mac builds yet. r?gps

https://reviewboard.mozilla.org/r/16871/#review15189

Ship It!
(In reply to Gregory Szorc [:gps] from comment #57)
> Comment on attachment 8651202 [details]
> MozReview Request: bug 935237 - add genisoimage+dmg binaries to tooltool
> manifest, use them in cross-mac mozconfig. r?gps
> 
> https://reviewboard.mozilla.org/r/16867/#review15187
> 
> ::: build/macosx/cross-mozconfig.common:38
> (Diff revision 1)
> > +export GENISOIMAGE=$topsrcdir/genisoimage/genisoimage
> > +export DMG_TOOL=$topsrcdir/dmg/dmg
> 
> Is there not a better place to put these? I'm not a huge fan of littering
> topsrcdir with untracked files.

That's where tooltool puts files.
(Assignee)

Comment 60

2 years ago
https://reviewboard.mozilla.org/r/16867/#review15187

> Is there not a better place to put these? I'm not a huge fan of littering topsrcdir with untracked files.

As glandium says, this is where tooltool unpacks files. It's certainly not what I would have chosen, but fixing that is probably out of scope for this bug.
(Assignee)

Comment 61

2 years ago
I had to update the binaries that this patch references because the desktop-build docker image that Taskcluster builds are using switched to CentOS 6, so I needed to rebuild my build tools to run there.
https://hg.mozilla.org/mozilla-central/rev/4e21632942b8
https://hg.mozilla.org/mozilla-central/rev/19bd24756050
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox43: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla43

Comment 64

2 years ago
Possible regression from current behavior? Can anyone confirm?

Test:
* Open the Firefox DMG created with genisoimage.
* Drag Firefox app to Desktop.
* Drag Firefox app from Desktop to Trash (or just delete)
* <regression> Security popup requiring user to enter their password

I believe this happens because the iso is umask 222 and when copied files retain read-only.

Solution I found was to create a hfs+ dmg image (instead of a iso dmg):

    dd if=/dev/zero of=uncompressed.dmg bs=1M count=256
    mkfs.hfsplus -v "Firefox 40.x" uncompressed.dmg
    hfsplus attract.dmg addall stagedir
    dmg dmg uncompressed.dmg compressed.dmg

Comment 65

2 years ago
One other thing to note, by using an hfs+ image it's straight-forward to set the custom icon attribute using hfsplus.

    hfsplus uncompressed.dmg attr / C

Also, attract.dmg in the last comment should have been uncompressed.dmg.
(Assignee)

Comment 66

2 years ago
Thanks, I had worked that out (I filed bug 1197325 to track it), but I already had the genisoimage way working so I decided just to land it.

RE: Permissions, are you saying the files inside the ISO are readonly?

Comment 67

2 years ago
Yes, all the files are read-only.
You need to log in before you can comment on or make changes to this bug.