Closed Bug 1690344 Opened 4 years ago Closed 4 years ago

nightly on Linux not picking up new updates before restarting

Categories

(Toolkit :: Application Update, defect)

defect

Tracking

()

RESOLVED INVALID

People

(Reporter: bhearsum, Unassigned)

Details

With https://bugzilla.mozilla.org/show_bug.cgi?id=353804 fixed, I would expect my Nightly to pick up updates without a need for a restart.

I haven't restarted my Nightly (currently 20210129095945) since Friday. My appdir currently has 20210130194115 in its updated dir, and as far as I can tell, no other update has been attempted.

A forced update check results in:

Cc["@mozilla.org/updates/update-service;1"].getService(Ci.nsITimerCallback).notify(null);
AUS:SVC getCanApplyUpdates - testing write access /home/bhearsum/usr/firefox-tl-nightly/update.test
AUS:SVC shouldUseService - returning false
AUS:SVC getCanApplyUpdates - able to apply updates
AUS:SVC Checker:getUpdateURL - update URL: https://aus5.mozilla.org/update/6/Firefox/87.0a1/20210129095945/Linux_x86_64-gcc3/tl/nightly/Linux%205.8.0-36-generic%20(GTK%203.24.23%2Clibpulse%2013.99.0)/ISET:SSE4_2,MEM:15884/default/default/update.xml
AUS:SVC UpdateService.canCheckForUpdates - able to check for updates
AUS:SVC Checker: checkForUpdates, force: false
AUS:SVC UpdateService.canCheckForUpdates - able to check for updates
AUS:SVC Checker:getUpdateURL - update URL: https://aus5.mozilla.org/update/6/Firefox/87.0a1/20210129095945/Linux_x86_64-gcc3/tl/nightly/Linux%205.8.0-36-generic%20(GTK%203.24.23%2Clibpulse%2013.99.0)/ISET:SSE4_2,MEM:15884/default/default/update.xml
AUS:SVC Checker:checkForUpdates - sending request to: https://aus5.mozilla.org/update/6/Firefox/87.0a1/20210129095945/Linux_x86_64-gcc3/tl/nightly/Linux%205.8.0-36-generic%20(GTK%203.24.23%2Clibpulse%2013.99.0)/ISET:SSE4_2,MEM:15884/default/default/update.xml
undefined
XHRGEThttps://aus5.mozilla.org/update/6/Firefox/87.0a1/20210129095945/Linux_x86_64-gcc3/tl/nightly/Linux%205.8.0-36-generic%20(GTK%203.24.23%2Clibpulse%2013.99.0)/ISET:SSE4_2,MEM:15884/default/default/update.xml
[HTTP/2 200 OK 409ms]

AUS:SVC Checker:onLoad - request completed downloading document
AUS:SVC Checker:onLoad - Getting sslStatus failed.
AUS:SVC Checker:onLoad - number of updates available: 1
AUS:SVC getCanApplyUpdates - testing write access /home/bhearsum/usr/firefox-tl-nightly/update.test
AUS:SVC shouldUseService - returning false
AUS:SVC getCanApplyUpdates - able to apply updates
AUS:SVC UpdateService:_selectAndInstallUpdate - download the update
AUS:SVC Creating Downloader
AUS:SVC UpdateService:_downloadUpdate
AUS:SVC Downloader:_selectPatch - not selecting a complete patch because this is not the first download of the session
AUS:SVC Downloader:downloadUpdate - no patch to download
AUS:SVC readStatusFile - status: applied, path: /home/bhearsum/usr/firefox-tl-nightly/updates/0/update.status

I checked the aus5 URL, and it's returning:

<updates>
<update type="minor" displayVersion="87.0a1" appVersion="87.0a1" platformVersion="87.0a1" buildID="20210202095107">
<patch type="complete" URL="https://archive.mozilla.org/pub/firefox/nightly/2021/02/2021-02-02-09-51-07-mozilla-central-l10n/firefox-87.0a1.tl.linux-x86_64.complete.mar" hashFunction="sha512" hashValue="b8874a8388a9e363cd186f0d790421a6dae06048b7393ac967aface9f20d20d364423c61b8b029455aa4cd4741c8050d4fb7a604bfed5c71a58fea486bde9c3d" size="68268890"/>
</update>
</updates>

So there's clearly a newer update available than the one that is currently staged.

Ok, judging from the logs, it does not appear that anything unexpected is going on here. Let me explain. I'm going to start from the beginning so that this bug is clear for anyone that is interested.

There are two bits of important background here:

First, updates come in two types: partial and complete. A partial update describes the changes between two Firefox version. Because of this, it's much smaller, but only valid when updating from a particular version. A complete update basically contains the entirety of Firefox, so it's much larger. Partial patches are only made for the last few versions, so once Firefox is sufficiently out of date, only complete patches will be available.

The second thing has to do with (the lack of) update chaining. Firefox never applies more than one patch at once. It doesn't download a whole bunch of sequential patches and then apply them in order. It downloads a newer patch to apply to the version that is currently installed. Then it can just get rid of the old patch and replace it with the newer one.

The consequence of these two things is that once Firefox is sufficiently out of date, every new update will be in the form of a large complete patch. I thought that downloading many complete patches might be an undesirable use of bandwidth (both for the user and for Mozilla). This is why Firefox treats complete patches differently. It won't download a complete patch if any patch was already downloaded during the current Firefox session. It also won't download any other patch once a complete patch has been downloaded.


Since your update XML has no partial patch, Firefox stopped downloading updates to wait for the current version to be installed, rather than downloading a new complete patch every 12 hours or so. Since you aren't far out of date, I'm not entirely sure why you are only getting served a complete patch. I'm guessing it has something to do with the recent version number bump?

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → INVALID

(In reply to Kirk Steuber (he/him) [:bytesized] from comment #1)

Ok, judging from the logs, it does not appear that anything unexpected is going on here. Let me explain. I'm going to start from the beginning so that this bug is clear for anyone that is interested.

There are two bits of important background here:

First, updates come in two types: partial and complete. A partial update describes the changes between two Firefox version. Because of this, it's much smaller, but only valid when updating from a particular version. A complete update basically contains the entirety of Firefox, so it's much larger. Partial patches are only made for the last few versions, so once Firefox is sufficiently out of date, only complete patches will be available.

The second thing has to do with (the lack of) update chaining. Firefox never applies more than one patch at once. It doesn't download a whole bunch of sequential patches and then apply them in order. It downloads a newer patch to apply to the version that is currently installed. Then it can just get rid of the old patch and replace it with the newer one.

The consequence of these two things is that once Firefox is sufficiently out of date, every new update will be in the form of a large complete patch. I thought that downloading many complete patches might be an undesirable use of bandwidth (both for the user and for Mozilla). This is why Firefox treats complete patches differently. It won't download a complete patch if any patch was already downloaded during the current Firefox session. It also won't download any other patch once a complete patch has been downloaded.

Thanks for the detailed explanation -- I didn't realize this was the expected behaviour.


Since your update XML has no partial patch, Firefox stopped downloading updates to wait for the current version to be installed, rather than downloading a new complete patch every 12 hours or so. Since you aren't far out of date, I'm not entirely sure why you are only getting served a complete patch. I'm guessing it has something to do with the recent version number bump?

I work on the update server and I can answer this part 😛. We always serve a complete update so long as both of the following are true:

  1. The version on the update is greater than the version requesting the update OR the version is the same and the buildid is greater
  2. Updates haven't been blocked entirely for some reason like platform deprecation.

We don't generate partials too far back for Nightly, for 2021-02-02-09-51-07 we only have partials for:

  • 20210131095227-20210202095107
  • 20210131211743-20210202095107
  • 20210201094443-20210202095107
  • 20210201213620-20210202095107

So 2021-01-29-09-59-45 would need a complete update. I expect this makes sense given how many Nightly builds there are and how few users need to get them. Betas partials also don't go back too far, it looks like we do the last 3 betas. Releases go back several versions, 85 has partials back to 83.

(Sorry for linking into the archive for the examples, I'm not sure where the logic actually resides for this.)

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #3)

We don't generate partials too far back for Nightly, for 2021-02-02-09-51-07 we only have partials for:

  • 20210131095227-20210202095107
  • 20210131211743-20210202095107
  • 20210201094443-20210202095107
  • 20210201213620-20210202095107

So 2021-01-29-09-59-45 would need a complete update. I expect this makes sense given how many Nightly builds there are and how few users need to get them. Betas partials also don't go back too far, it looks like we do the last 3 betas. Releases go back several versions, 85 has partials back to 83.

(Sorry for linking into the archive for the examples, I'm not sure where the logic actually resides for this.)

No worries, I appreciate the explanation! For the record, I believe this comes from https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/util/partials.py#203 for nightly.

You need to log in before you can comment on or make changes to this bug.