Incorrect XPI hash on Windows

RESOLVED FIXED

Status

Webtools
Telemetry Server
RESOLVED FIXED
6 months ago
5 months ago

People

(Reporter: ashughes, Assigned: ashughes)

Tracking

Trunk

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

While trying to develop and deploy a Telemetry Experiment on Windows I kept encountering an error when testing installation of my addon. After much debugging and troubleshooting with Felipe Gomes we determined that build.py was generating a different sha256 hash than what was expected.

Upon further investigation, with help from Jukka Jylanki, we discovered that build.py is loading the file in text-mode instead of binary mode. 

Further explanation of the issue provided by Jukka:
> On Linux and OS X, there is not much of a concept of a binary vs text file, 
> which is why the issue didn't manifest there, but on Windows, file opening 
> is done either in text mode or binary mode, and especially when hashing, all 
> files (even text files) need to be opened in binary mode so that the hashing 
> method correctly hashes the source bits untransformed.
> 
> The bad transformation that occurred here on Windows is that if a file is 
> opened in text mode, the bytes representing Windows line endings on disk 
> (\r\n or 0x0D 0x0A) will get loaded up to memory as just a single byte 
> \n (0x0A), which would change the original file data that is getting hashed. >
> In the case of the binary compressed .xpi file, it looks like the file just 
> accidentally happens to have a byte sequence 0x0D 0x0A, which got mutated, 
> and hence the hash being different.

The fix is to simply make sure we always load the file for reading as binary. I have a patch to fix this.
Created attachment 8818347 [details] [diff] [review]
bug1323304_v1.patch
Attachment #8818347 - Flags: review?(felipc)
Comment on attachment 8818347 [details] [diff] [review]
bug1323304_v1.patch

Review of attachment 8818347 [details] [diff] [review]:
-----------------------------------------------------------------

\o/ so glad you find this!
Attachment #8818347 - Flags: review?(felipc) → review+
Blocks: 1319206
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/38ce1b1eea2ce396b3566c878f4f79da3fcbc547
Status: ASSIGNED → RESOLVED
Last Resolved: 5 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.