1816990 - Teach `libdmg-hfsplus` to produce "attributable" DMGs

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Nick Alexander :nalexander [he/him]]

For "DMG extended attributes" attribution, we want to write DMGs that are optimized for modifying a small block of placeholder bytes without fully parsing the DMG and/or the underlying HFS+ filesystem. I propose to implement this in https://github.com/mozilla/libdmg-hfsplus.

I am envisioning a new command, like:

dmg attributable Firefox.dmg Attributable.dmg -placeholderStart '{"mozilla-attribution":' -placeHolderLength 512

This will:

• read an existing DMG
• recompress it, keeping the place holder bytes in an uncompressed BZ_RAW block
• strip extraneous resource blocks, including the "cSum" block which has a non-standard checksum
• add a resource block with data that allows to update the placeholder bytes: offsets, lengths, partial CRC32 checksums, etc.

Comment 1

[[e-mail] (:bhearsum)]

I've been doing some additional investigating and testing here. I noticed a few things that will need some tweaking:

• strip extraneous resource blocks, including the "cSum" block which has a non-standard checksum

I discovered that any DMGs that have cSum removed cannot be opened by Finder. (hdiutil still mounts them - but obviously we cannot ship in that state.) However, it appears that cSum is not verified beyond whether or not it is present (more on this below).

I also found that adding any resource blocks beyond what exist by default (blk, cSum, nsiz, size, plst) also cause the DMG to be unopenable by Finder.

With all of this in mind, I made a few tweaks to Nick's libdmg-hfsplus branch to:

• Stop stripping cSum, et. al
• Put the attribution data in the unused Name key in the plst object instead of its own Resource

Verification

Attributable & Attributed DMGs

I built two DMGs with the above patch:

• An attributable DMG, where Firefox.app inside of it contained a com.apple.application-instance extended attribute, and dmg build was given a sentinel value:

dd if=/dev/zero of=attributable.hfs bs=1M count=380
./newfs_hfs -v Firefox attributable.hfs
hfsplus attributable.hfs addall stage # the `stage` directory contained the 116.0 Firefox.app release build
hfsplus attributable.hfs setattr "Firefox.app" com.apple.application-instance __MOZILLA__attribution_sentinel_placeholder__
dmg build attributable.hfs attributable.dmg __MOZILLA__attribution_sentinel_placeholder__
dmg attribute attributable.dmg attributed.dmg __MOZILLA__attribution_sentinel_placeholder__ __MOZILLA_updated_attribution_data__

• An attributed DMG, with updated attribution data:

dmg attribute attributable.dmg attributed.dmg __MOZILLA__attribution_sentinel_placeholder__ __MOZILLA_updated_attribution_data__

Both of the resulting DMGs mounted just fine with Finder, and I was able to pull the expected attribution string in Firefox with:

new TextDecoder("utf-8").decode(await IOUtils.getMacXAttr("/Volumes/Firefox/Firefox.app", "com.apple.application-instance"))

(The attributed one actually ended up showing __MOZILLA_updated_attribution_data__eholder__ - but that likely just means that updating attribution is not nul terminating strings or otherwise zero'ing out the block - nothing that disproves the results.)

I also had QA run through this test on all supported versions of macOS - and they had the same results.

The attributed DMG most certainly has modified blkx data in it, which means the cSum data is no longer accurate - suggesting that it is not verified at all.

cSum is not verified?

To test the theory that cSum data is not validated, I tweaked libdmg-hfsplus to override some of the cSum data as it is being written out. This should mean that even DMGs built without attribution have a wrong cSum:

diff --git a/dmg/resources.c b/dmg/resources.c
index 45db951..1df5fe8 100644
--- a/dmg/resources.c
+++ b/dmg/resources.c
@@ -729,16 +729,26 @@ static void writeResourceData(AbstractFile* file, ResourceData* data, ResourceKe
 
   if(strcmp((char*) curResource->key, "blkx") == 0)
     abstractFilePrint(file, "%s\t<key>CFName</key>\n%s\t<string>%s</string>\n", tabs, tabs, data->name);
 
   abstractFilePrint(file, "%s\t<key>Data</key>\n%s\t<data>\n", tabs, tabs);
   
   if(flipData) {
     dataBuf = (unsigned char*) malloc(data->dataLength);
+    if (strcmp((char*) curResource->key, "cSum") == 0) {
+      data->data[0] = 'b';
+      data->data[1] = 'h';
+      data->data[2] = 'e';
+      data->data[3] = 'a';
+      data->data[4] = 'r';
+      data->data[5] = 's';
+      data->data[6] = 'u';
+      data->data[7] = 'm';
+    }
     memcpy(dataBuf, data->data, data->dataLength);
     (*flipData)(dataBuf, 1);
     writeBase64(file, dataBuf, data->dataLength, tabLength + 1, 43);
     free(dataBuf);
   } else {
     writeBase64(file, data->data, data->dataLength, tabLength + 1, 43);
   }

A DMG built without any attribution (or even extended attribute on Firefox.app) mounted just fine in Finder, which, unless I'm missing something, proves that cSum is not validated.

Comment 2

[[e-mail] (:bhearsum)]

This support was merged in https://github.com/mozilla/libdmg-hfsplus/pull/6. I'll have a revision up shortly to bump the version we're using in mozilla-central.

Comment 3

[[e-mail] (:bhearsum)]

Attachment: Bug 1816990: update libdmg-hfsplus revision to pick up support for attributable DMGs r?#releng-reviewers!

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c65aa3dce75b
update libdmg-hfsplus revision to pick up support for attributable DMGs r=releng-reviewers,hneiva

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/c65aa3dce75b