1073493 - Update failure: "The integrity of the update could not be verified" for the complete mar file

Status: RESOLVED INCOMPLETE

(Toolkit :: Application Update, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Recently I have seen a couple of those integrity problems while updating Firefox. Mainly by testing our refactored update tests for Mozmill. I already mentioned it to Nick earlier this week, but then I was not able to provide a useful HTTP log. But now I have one. Here the details...

Console output of the updater:

*** AUS:SVC Downloader:onStopRequest - download verification failed
*** AUS:SVC getStatusTextFromCode - transfer error: The integrity of the update could not be verified, code: verification_failed

HTTP log for the whole update process:
https://www.dropbox.com/s/hah73kgzrq1s50h/log-integrity.zip?dl=0

When you check the log file you want to query for the download of the complete mar file.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

It looks like that the last chunk we request is:

2014-09-26 11:55:57.401154 UTC - -1842243712[7f7790e634a0]:   Range: bytes=8700000-
2014-09-26 11:55:57.401157 UTC - -1842243712[7f7790e634a0]:   If-Range: "4874170-32d49fa-503e245bea46f"

There is not a single http response, but hundreds of the following lines:

2014-09-26 11:55:57.812619 UTC - -1842243712[7f7790e634a0]: nsHttpChannel::OnDataAvailable [this=7f7751069800 request=7f7752750980 offset=205390 count=1440]

Whereby the count is mostly 1440, but also varies.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Patrick, I think that this is also related to a networking issue. Would you mind to have a look at the HTTP log?

Comment 3

[Robert Strong (they/them - no direct email)]

Did the bug also go away after the new Firefox Nightly came out as it did for bug 816472?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This happened only once to me. So I cannot give a trustworthy answer to this question. Sorry.

Comment 5

[Patrick McManus [:mcmanus]]

i'm going to cancel the needinfo here presuming that this is the same kind of analysis that bug 1073459 needed.. over there we saw that the server could give the wrong byte ranges and that certainly would make the checksum fail

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Patrick, based on your investigation on the other bug I have taken a look at this one. What I found looks suspicious too:

Request:
2014-09-26 11:55:57.401154 UTC - -1842243712[7f7790e634a0]:   Range: bytes=8700000-

Response:
2014-09-26 11:55:57.602008 UTC - 2062546688[7f7790e63920]:   Content-Length: 44599706
2014-09-26 11:55:57.602012 UTC - 2062546688[7f7790e63920]:   Content-Range: bytes 8700000-53299705/53299706
2014-09-26 11:55:57.602022 UTC - 2062546688[7f7790e63920]:   X-Backend-Server: ftp5.dmz.scl3.mozilla.com

Please note the 1 byte difference here: 53299705/53299706. I assume that this causes the downloaded file to be 1 byte smaller than expected which let the checksum fail. Not sure what's up with the last byte.

Comment 7

[Patrick McManus [:mcmanus]]

the 1 byte difference is not the issue - the range counting just starts at 0. So 0-1/2 is a the full range of a 2 byte file.

Comment 8

[Robert Strong (they/them - no direct email)]

We didn't see any user reports of verification failures and we no longer check the hashes of the mar files so resolving as incomplete since there is nothing else we can do here.