1584826 - Both partial and full Nightly updates fail

Status: RESOLVED DUPLICATE of bug 1579415

(Release Engineering :: Release Automation, defect)

Description

[Francesco Lodolo [:flod]]

I'm on Nightly (71.0a1 (2019-09-28), macOS Italian.

I get a notification that an update is available, but it fails and prompt me to download a new version of Nightly from mozilla.org

The main culprit in the log seems to be maxProgress: 64475685 is not equal to expected patch size: 64475725. Still not sure why it doesn't download the full build though.

The full update log

AUS:SVC Checker: checkForUpdates, force: true
AUS:SVC UpdateService.canCheckForUpdates - able to check for updates
AUS:SVC Checker:getUpdateURL - update URL: https://aus5.mozilla.org/update/6/Firefox/71.0a1/20190928094605/Darwin_x86_64-gcc3/it/nightly/Darwin%2018.7.0/ISET:SSE4_2,MEM:16384/default/default/update.xml?force=1
AUS:SVC Checker:checkForUpdates - sending request to: https://aus5.mozilla.org/update/6/Firefox/71.0a1/20190928094605/Darwin_x86_64-gcc3/it/nightly/Darwin%2018.7.0/ISET:SSE4_2,MEM:16384/default/default/update.xml?force=1
AUS:SVC Checker:onLoad - request completed downloading document
AUS:SVC Checker:onLoad - Getting sslStatus failed.
AUS:SVC Checker:onLoad - number of updates available: 1
AUS:SVC getElevationRequired - recursively testing write access on /Applications/FirefoxNightly.app
AUS:SVC getElevationRequired - able to write to application bundle, elevation not required
AUS:SVC getCanApplyUpdates - testing write access /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/update.test
AUS:SVC getCanApplyUpdates - bypass the write since elevation can be used on Mac OS X
AUS:SVC Creating Downloader
AUS:SVC UpdateService:_downloadUpdate
AUS:SVC readStringFromFile - file doesn't exist: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/updates/0/update.status
AUS:SVC readStatusFile - status: null, path: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/updates/0/update.status
AUS:SVC getCanUseBits - Not using BITS because this is not Windows
AUS:SVC Downloader:downloadUpdate - Starting nsIIncrementalDownload with url: https://archive.mozilla.org/pub/firefox/nightly/partials/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-mozilla-central-71.0a1-mac-it-20190928094605-20190928211735.partial.mar, path: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/updates/0/update.mar, interval: 0
AUS:SVC Downloader:onStartRequest - original URI spec: https://archive.mozilla.org/pub/firefox/nightly/partials/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-mozilla-central-71.0a1-mac-it-20190928094605-20190928211735.partial.mar, final URI spec: https://archive.mozilla.org/pub/firefox/nightly/partials/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-mozilla-central-71.0a1-mac-it-20190928094605-20190928211735.partial.mar
AUS:SVC Downloader:onProgress - progress: 16800/6464036
AUS:SVC Downloader:onProgress - maxProgress: 6464036 is not equal to expected patch size: 6470876
AUS:SVC Downloader: cancel
AUS:SVC Downloader:onProgress - progress: 0/6464036
AUS:SVC Downloader:onStopRequest - downloader: nsIIncrementalDownload, original URI spec: https://archive.mozilla.org/pub/firefox/nightly/partials/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-mozilla-central-71.0a1-mac-it-20190928094605-20190928211735.partial.mar, final URI spec: https://archive.mozilla.org/pub/firefox/nightly/partials/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-mozilla-central-71.0a1-mac-it-20190928094605-20190928211735.partial.mar, status: 2147549183
AUS:SVC Downloader:onStopRequest - status: 2147549183, current fail: 0, max fail: 10, retryTimeout: 2000
AUS:SVC Downloader:onStopRequest - non-verification failure
AUS:SVC getStatusTextFromCode - transfer error: Operazione non riuscita (motivo sconosciuto), default code: 2152398849
AUS:SVC Downloader:onStopRequest - setting state to: download-failed
AUS:SVC Downloader:onStopRequest - verification of patch failed, downloading complete update patch
AUS:SVC UpdateService:_downloadUpdate
AUS:SVC readStringFromFile - file doesn't exist: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/updates/0/update.status
AUS:SVC readStatusFile - status: null, path: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/updates/0/update.status
AUS:SVC Downloader:_selectPatch - found existing patch with state: null
AUS:SVC getCanUseBits - Not using BITS because this is not Windows
AUS:SVC Downloader:downloadUpdate - Starting nsIIncrementalDownload with url: https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-71.0a1.it.mac.complete.mar, path: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/updates/0/update.mar, interval: 0
AUS:SVC Downloader:onStartRequest - original URI spec: https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-71.0a1.it.mac.complete.mar, final URI spec: https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-71.0a1.it.mac.complete.mar
AUS:SVC Downloader:onProgress - progress: 8497/64475685
AUS:SVC Downloader:onProgress - maxProgress: 64475685 is not equal to expected patch size: 64475725
AUS:SVC Downloader: cancel
AUS:SVC Downloader:onProgress - progress: 0/64475685
AUS:SVC Downloader:onStopRequest - downloader: nsIIncrementalDownload, original URI spec: https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-71.0a1.it.mac.complete.mar, final URI spec: https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-28-21-17-35-mozilla-central-l10n/firefox-71.0a1.it.mac.complete.mar, status: 2147549183
AUS:SVC Downloader:onStopRequest - status: 2147549183, current fail: 0, max fail: 10, retryTimeout: 2000
AUS:SVC Downloader:onStopRequest - non-verification failure
AUS:SVC getStatusTextFromCode - transfer error: Operazione non riuscita (motivo sconosciuto), default code: 2152398849
AUS:SVC Downloader:onStopRequest - setting state to: download-failed
AUS:SVC Downloader:onStopRequest - notifying observers of error. topic: update-error, status: download-attempts-exceeded, downloadAttempts: 9 maxAttempts: 2
AUS:SVC UpdateManager:_writeUpdatesToXMLFile - no updates to write. removing file: /Users/flodolo/Library/Caches/Mozilla/updates/Applications/FirefoxNightly/active-update.xml

Comment 1

[Francesco Lodolo [:flod]]

Same Nightly, different machine, just downloaded the full build (unfortunately I didn't have the update log enabled).

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Looking at https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=nightly&revision=ca5c8cdce0c9af1ccc1afe0a0cec0a5ed97d4b78, seems like yet again we triggered two nightlies off the same revision on mozilla-central which confused Balrog as same locale/platform pairs have different blobs sizes. This is likely a dupe of bug 1579415.

I'll raise this again in RelEng, thanks for filing :flod!

Comment 3

[Mihai Tabara [:mtabara]⌚️GMT]

(In reply to Francesco Lodolo [:flod] from comment #1)

Same Nightly, different machine, just downloaded the full build (unfortunately I didn't have the update log enabled).

FTR, updates should work smoothly now, there's https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=nightly&revision=ca257801d86f5d4e9c1768722c7131216c07d38f ongoing with a different revision.