1323304 - Incorrect XPI hash on Windows

Status: RESOLVED FIXED

(Webtools Graveyard :: Telemetry Server, defect)

Description

[u279076]

While trying to develop and deploy a Telemetry Experiment on Windows I kept encountering an error when testing installation of my addon. After much debugging and troubleshooting with Felipe Gomes we determined that build.py was generating a different sha256 hash than what was expected.

Upon further investigation, with help from Jukka Jylanki, we discovered that build.py is loading the file in text-mode instead of binary mode. 

Further explanation of the issue provided by Jukka:
> On Linux and OS X, there is not much of a concept of a binary vs text file, 
> which is why the issue didn't manifest there, but on Windows, file opening 
> is done either in text mode or binary mode, and especially when hashing, all 
> files (even text files) need to be opened in binary mode so that the hashing 
> method correctly hashes the source bits untransformed.
> 
> The bad transformation that occurred here on Windows is that if a file is 
> opened in text mode, the bytes representing Windows line endings on disk 
> (\r\n or 0x0D 0x0A) will get loaded up to memory as just a single byte 
> \n (0x0A), which would change the original file data that is getting hashed. >
> In the case of the binary compressed .xpi file, it looks like the file just 
> accidentally happens to have a byte sequence 0x0D 0x0A, which got mutated, 
> and hence the hash being different.

The fix is to simply make sure we always load the file for reading as binary. I have a patch to fix this.

Comment 1

[u279076]

Attachment: bug1323304_v1.patch

Comment 2

[:Felipe Gomes (needinfo for replies!)]

Comment on attachment 8818347 [details] [diff] [review]
bug1323304_v1.patch

Review of attachment 8818347 [details] [diff] [review]:
-----------------------------------------------------------------

\o/ so glad you find this!

Comment 3

[:Felipe Gomes (needinfo for replies!)]

https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/38ce1b1eea2ce396b3566c878f4f79da3fcbc547