Firefox Nightly does not update with certain distribution.ini values
Categories
(Toolkit :: Application Update, defect)
Tracking
()
People
(Reporter: carm_carm, Assigned: bytesized)
Details
(Keywords: regression)
Attachments
(2 files)
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Steps to reproduce:
Install a Firefox nightly from an offline installer that is several days old. DO NOT launch it. Create a distribution.ini file in the correct directory with any values. IN the GLOBAL section leave the id, version and about blank. i.e (included attachment )
[Global]
id=
version=
about=
Launch Firefox Nightly in a new profile.
Go to settings - help - about nightly and try to update. it show as up to date despite being several days old.
Now add something, anything in those values like so:
[Global]
id=Custon-Build
version=0
about=customized for privacy
I have been using the values being blank for about a year, then November/December 2019 Firefox would not update anymore. I tried literally commenting out every other setting until I got to global. Leaving globals' blank causes it to fail update every time
The same setting in release are unaffected and works just fine
Actual results:
Firefox Nightly showed up to date despite having a build date of several days old
Expected results:
Firefox should have found a new build and updated.
Issue HAS moved to release 73.0.1 on today's date - NO way to edit additions/comments
|
||
Comment 3•4 years ago
|
||
Moving to the right bugzilla component, and also ask for a regression range (as it seems based on comment 0 that this may have regressed around Nov/Dec 2019).
|
Assignee | |
Comment 4•4 years ago
|
||
I'm taking a look at a Nightly build from long before the problem is said to have started (A 2019-01-01 build, version 66.0a1, downloaded here). And I can definitely see the cause.
This is the update URL format. The last two variables being substituted are read from distribution.ini.
https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml
This is the update URL I get with no distribution.ini file at all:
https://aus5.mozilla.org/update/6/Firefox/66.0a1/20190101094742/WINNT_x86_64-msvc-x64/en-US/nightly/Windows_NT%2010.0.0.0.18362.657%20(x64)/ISET:SSE4_2,MEM:16192/default/default/update.xml (Link)
And this is the one I get with a distribution.ini with blank Global.id, Global.version, and Global.about values:
https://aus5.mozilla.org/update/6/Firefox/66.0a1/20190101094742/WINNT_x86_64-msvc-x64/en-US/nightly/Windows_NT%2010.0.0.0.18362.657%20(x64)/ISET:SSE4_2,MEM:16192///update.xml (Link)
If you click on the link for each of these update XMLs, you will see that the first URL results in an XML describing an update, while the second URL results in an XML indicating that there are no updates. Since I can reproduce this prior to the stated introduction of the bug, I'm suspicious that this might be the result of a change to the update backend (Balrog). Perhaps it used to handle that second URL nicely but it doesn't any longer?
@bhearsum Do you know if that is a possible explanation for the reported change in behavior?
|
||
Comment 5•4 years ago
|
||
(In reply to Kirk Steuber (he/him) [:bytesized] from comment #4)
I'm taking a look at a Nightly build from long before the problem is said to have started (A 2019-01-01 build, version 66.0a1, downloaded here). And I can definitely see the cause.
This is the update URL format. The last two variables being substituted are read from distribution.ini.
https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml
This is the update URL I get with no distribution.ini file at all:
https://aus5.mozilla.org/update/6/Firefox/66.0a1/20190101094742/WINNT_x86_64-msvc-x64/en-US/nightly/Windows_NT%2010.0.0.0.18362.657%20(x64)/ISET:SSE4_2,MEM:16192/default/default/update.xml(Link)
And this is the one I get with a distribution.ini with blank Global.id, Global.version, and Global.about values:
https://aus5.mozilla.org/update/6/Firefox/66.0a1/20190101094742/WINNT_x86_64-msvc-x64/en-US/nightly/Windows_NT%2010.0.0.0.18362.657%20(x64)/ISET:SSE4_2,MEM:16192///update.xml(Link)If you click on the link for each of these update XMLs, you will see that the first URL results in an XML describing an update, while the second URL results in an XML indicating that there are no updates. Since I can reproduce this prior to the stated introduction of the bug, I'm suspicious that this might be the result of a change to the update backend (Balrog). Perhaps it used to handle that second URL nicely but it doesn't any longer?
@bhearsum Do you know if that is a possible explanation for the reported change in behavior?
As far as I know, Balrog has never support a section of an URL being entirely empty like that, and it certainly wasn't ever the intent to. I tried this with an older version of Balrog locally (code from June 2019), and this behaviour still exists at that point. I don't see anything on the Balrog side that would've caused this.
Adding mkaply, because he knows the partner distribution side of things very well.
|
||
Comment 6•4 years ago
|
||
Empty distribution values are invalid and shouldn't have been added to the update URL. my gut says that something changed about how we do URL substitution.
Just to be clear, this only happens in nightly? Not in release?
Might I ask why you have a distribution.ini with empty values?
|
||
Comment 7•4 years ago
|
||
Bugbug thinks this bug is a regression, but please revert this change in case of error.
|
|
||
Comment 8•4 years ago
|
||
I just ran a quick check on Firefox 68 and it does blank values for distribution.ini and no updates are available.
I'm checking old nightlies now
|
|
||
Comment 9•4 years ago
|
||
I just verified a nightly from mid last year also doesn't show updates.
So if this did regress, something would have changed in balrog.
I don't know whether to mark this as invalid or fix the URL code to handle empty distribution IDs...
|
|
Assignee | |
Comment 10•4 years ago
|
||
I don't know whether empty distribution IDs is something we technically support, but I would like update to be as robust as possible, and this should be a fairly simple change. So I'm going to go ahead and post a patch.
|
|
Assignee | |
Comment 11•4 years ago
|
||
This patch prevents the URL formatter from setting empty values for distribution data. It also prevents any non-url-encoded value from being substituted into the URL.
|
||
Updated•4 years ago
|
|
||
Comment 12•4 years ago
|
||
Pushed by ksteuber@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/353752111b71 Make the Update URL formatter more robust r=mhowell,bhearsum,mkaply
|
|
||
Comment 13•4 years ago
|
||
Pushed by cbrindusan@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/16487cdee6ee Fixed Eslint error on UpdateUtils.jsm. CLOSED TREE
|
||
Comment 14•4 years ago
|
||
Backed out 2 changesets (bug 1615136) for xpcshell failures at toolkit/modules/tests/xpcshell/test_UpdateUtils_url.js
Backout: https://hg.mozilla.org/integration/autoland/rev/0aed72c7e87da0818f8062641b7126b84aa9b5b6
Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=353752111b7125a283a95d4c2bb2a5bd246b3f26
Failure log:
Xpcshell: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=291701676&repo=autoland&lineNumber=3553
Eslint: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=291698881&repo=autoland&lineNumber=307
|
|
Assignee | |
Comment 16•4 years ago
|
||
I'm removing the URL encoding parts of my patch. I hadn't thought that URL-encoding each part of the path would cause any changes, but it turns out that the %SYSTEM_CAPABILITIES% replacement currently looks like ISET:SSE4_2,MEM:16192, which is not encoded properly. I don't want to make that change here, but I filed Bug 1620327 to address it. In the meantime, I'm going to push a patch that addresses only the issue at hand: the empty distribution values.
|
|
||
Comment 17•4 years ago
|
||
Pushed by ksteuber@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5b935c9b36ab Make the Update URL formatter more robust r=mhowell,bhearsum,mkaply
|
||
Comment 18•4 years ago
|
||
| bugherder | ||
|
||
Comment 19•4 years ago
|
||
Doesn't sound like we need to worry about backporting this to ESR, but LMK if that's incorrect.
|
||
Updated•4 years ago
|
|
||
Comment 20•4 years ago
|
||
Reproduced the initial issue using an old Nightly before the fix and verified that using a nightly build after the fix can update to latest Nightly when using a blank distribution.ini file across platforms (Windows 10 64bit, macOS 10.15 and Ubuntu 18.04 64bit). I also checked that Firefox beta 75 is not affected by this.
Description
•