711275 - Make sure appVersion changes with every Firefox 10 beta

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Robert Strong (they/them - no direct email)]

Per the following email thread we'd like to test the app update flow with the add-ons default to compatible feature. For app update, this will require changing appVersion in the update xml. For add-ons, Blair should be able to tell us if anything is required. We have a couple of people testing this but it would be good to also test this with the Aurora user base. It might also make sense to do this on Nightly instead or on both Nightly and Aurora.

------------------------------------------------------------------------

Hm, yes, that sounds like a worthwhile test. I remember we did something
similar for 4.0.

The automated tests already test that the Addons Manager correctly
determines compatibility for a newer version of the application than
that's running. But it doesn't explicitly test the whole process
(checking for dialogs, etc) in the context of DTC.

What needs to be done? Just file a RelEng bug? (Perhaps Rob should do
that, I don't have much background in app update stuff.)

- Blair



On 16/12/11 11:12, Robert Strong wrote:
> Hi Blair,
>
> Lawrence has it right but a little extra detail might help. The
> update.xml for app update advertises the app version the update is for
> but it can be different than the actual app version. If teleng changed
> this value to a higher app version it should test this feature during
> the app update process for users with builds that include the feature.
> Also, the actual update doesn't have to change the application's
> version. This would test whether the user was prompted for an app update
> or if it downloaded the update in the background based on whether their
> extensions were compatible by default. The add-ons manager
> incompatibility ui after app upgrade wouldn't be tested in this scenario
> unless the update's application version actually changed which is out of
> scope for this test and is more difficult.
>
> Cheers,
> Robert
>
> On 12/15/2011 1:39 PM, Lawrence Mandel wrote:
>> Hi Blair,
>>
>> Rob brought up a question today about testing the add-ons default to
>> compatible feature with Aurora/Beta users. Currently our Nightly and
>> Aurora users are updating to a later build but not a later version of
>> the browser. Rob suggested bumping the version in update.xml in order
>> to simulate users having a later version of the browser to confirm the
>> expected behaviour. (Rob, please correct me if I've got any of this
>> wrong.) Is this a test scenario that you've thought about?
>>
>> Rob mentioned specific interest in verifying that no dialog is shown
>> in this scenario.
>>
>> Thanks,
>>
>> Lawrence

Comment 1

[Blair McBride [:Unfocused] (UNAVAILABLE)]

(In reply to Robert Strong [:rstrong] (do not email) from comment #0)
> For add-ons, Blair should be able to tell us if anything is required. 

I can't think of anything.

Comment 2

[Lawrence Mandel [:lmandel] (use needinfo)]

We want to test this feature as described for Firefox 10 Beta, which starts this week. How quickly can this request be turned around?

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

To help us, could you please be more specific about what is wanted.

* Comment #0 talks about maybe aurora maybe nightly, maybe nightly and aurora 
* Comment #2 talks about beta, you want to test in beta or prior to ?
* We have version and extensionVersion for nightly/aurora, which we can figure out how to modify
* We have appVersion in release builds
* Could you give a specific example of what are you asking for ?

Comment 4

[Robert Strong (they/them - no direct email)]

I'm not to familiar with the feature itself and would like someone that is to decide where to test.

appVersion is the new extensionVersion so if you don't have appVersion use extensionVersion. Is there a bug or plans to use the new format everywhere?

Specifically, the appVersion or extensionVersion attribute value should be bumped to cause a check for incompatible add-ons. So, if it is 10.x changing it to 11.x would cause an incompatibility check.

Comment 5

[[e-mail] (:bhearsum)]

I'll look at this tomorrow.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

My suggestion is that we start saying 'extv=13.0a1' in the snippets for mozilla-central, which will show up as extensionVersion="13.0a1" in the update.xml. That will let users on Nighlty be guinea pigs, both sides of the merge/version bump on the 20th. I don't know how that plays with any other code which might already in place there affecting compatibility.

I don't think we can do Aurora at the moment because those updates are disabled around merge time. 

Not sure what you're intending for beta, but there is always the update test channels for QA to check between spinninng 10.0b1 and shipping it to users.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Nick Thomas [:nthomas] from comment #6)
> I don't think we can do Aurora at the moment because those updates are
> disabled around merge time. 

Not quite true. Better to say that 11.0a2 builds aren't published to the aurora channel for a few days, so we can only affect people updating to the last 10.0a2 aurora build. If that's a big proportion it might be useful.

Comment 8

[[e-mail] (:bhearsum)]

Just talked with Lawrence about this. What's needed here is to have Beta users, at each 10 Beta update, run through the new code paths. This needs to be done by 10.0b2, because the 10.0b1 jump will already have a version bump (9.0 -> 10.0). Need to figure out how to jam this into the automation, but it should otherwise be simple.

Comment 9

[[e-mail] (:bhearsum)]

Rail and I talked about how best to do this with our tools, and decided that it's probably easiest to post-process the snippets rather than trying to build this into patcher/buildbot. So, I'll need to write a script that does that, and have it optionally run as part of the updates process.

Comment 10

[[e-mail] (:bhearsum)]

Testing potential patches in staging:
https://github.com/bhearsum/tools/compare/master...use-version-for-snippets
https://github.com/bhearsum/buildbotcustom/compare/master...use-version-for-snippets

Comment 11

[[e-mail] (:bhearsum)]

Attachment: run edit-snippets.sh for Firefox 10 betas

Comment 12

[[e-mail] (:bhearsum)]

Attachment: tool for mass edit of snippets

This tool worked fine for my staging run, and should be generic enough for other mass edits that we have to do on occasion.

I did a staging run of this by changing the hardcoded version to 9.0b, and re-running 9.0b6 in staging. I compared the resulting snippets to the 9.0b6 snippets we shipped. With some bash, proof that only the appVersion changed:
diff -Naur /opt/aus2/snippets/staging/Firefox-9.0b6-build1/ Firefox-9.0b6-build1/ > production-vs-rerun.diff
[bhearsum@dm-ausstage01 ~]$ grep '^[-+]' production-vs-rerun.diff | grep -v '\-\-\-' | grep -v '+++' | sort | uniq
-appVersion=9.0
+appVersion=9.0b6

(Quick explanation: I did a full diff, grepped out context and diff headers, and used uniq to make sure every file had the same diff.)

Comment 13

[[e-mail] (:bhearsum)]

Just to be clear about exactly what the resulting snippets and XML looks like, here's sample snippets from my test run:
-bash-3.2$ cat complete.txt 
version=2
type=complete
url=http://download.mozilla.org/?product=firefox-9.0b6-complete&os=win&lang=en-US
hashFunction=SHA512
hashValue=4a4f1cebf4ab2392a645df5b2ad2ff19376ee6e21ca15210c78d95ef3072894f76cfb8bb9540b50a1faeba287b7b4ce29f53745ad1d1bded3d8a3f9a7e947732
size=20144723
build=20111212185108
displayVersion=9.0 Beta
appVersion=9.0b6
platformVersion=9.0
detailsUrl=https://www.mozilla.com/en-US/firefox/9.0/releasenotes/
actions=silent
-bash-3.2$ cat partial.txt 
version=2
type=partial
url=http://download.mozilla.org/?product=firefox-9.0b6-partial-9.0b5&os=win&lang=en-US
hashFunction=SHA512
hashValue=dc8eabcb090c185ccf685ed57b0f1b43b6c0e468dba81bac1a28c0a86c9d1d0ae5f444dbadeb2b39b28b89b414f191c7f055eec8d0fe75ec239a5e6602b8c717
size=2212029
build=20111212185108
displayVersion=9.0 Beta
appVersion=9.0b6
platformVersion=9.0
detailsUrl=https://www.mozilla.com/en-US/firefox/9.0/releasenotes/
actions=silent

And here's the XML they generate:
<updates><update type="minor" displayVersion="9.0 Beta" appVersion="9.0b6" platformVersion="9.0" buildID="20111212185108" detailsURL="https://www.mozilla.com/en-US/firefox/9.0/releasenotes/" actions="silent"><patch type="complete" URL="http://download.mozilla.org/?product=firefox-9.0b6-complete&os=win&lang=en-US&force=1" hashFunction="SHA512" hashValue="4a4f1cebf4ab2392a645df5b2ad2ff19376ee6e21ca15210c78d95ef3072894f76cfb8bb9540b50a1faeba287b7b4ce29f53745ad1d1bded3d8a3f9a7e947732" size="20144723"/><patch type="partial" URL="http://download.mozilla.org/?product=firefox-9.0b6-partial-9.0b5&os=win&lang=en-US&force=1" hashFunction="SHA512" hashValue="dc8eabcb090c185ccf685ed57b0f1b43b6c0e468dba81bac1a28c0a86c9d1d0ae5f444dbadeb2b39b28b89b414f191c7f055eec8d0fe75ec239a5e6602b8c717" size="2212029"/></update></updates>

Comment 14

[Lawrence Mandel [:lmandel] (use needinfo)]

Great work Ben.

Rob - Do you know if this change will result in any user visible change in the browser?

Comment 15

[Lawrence Mandel [:lmandel] (use needinfo)]

Virgil - Wanted to alert you of this change to assist in the testing of add-ons default to compatible.

Comment 16

[Armen [:armenzg]]

The patches that landed on "default" are now on production and the masters have been reconfigured.

Comment 17

[Robert Strong (they/them - no direct email)]

(In reply to Lawrence Mandel [:lmandel] from comment #14)
> Great work Ben.
> 
> Rob - Do you know if this change will result in any user visible change in
> the browser?
It doesn't change anything in the browser the user would typically ever see. It does change the update flow in that it will require the user to opt-in to the update if they have add-ons that are not compatible with the update and I think the update history which is available in preferences / options -> advanced -> update will display the version offered.

Comment 18

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to Robert Strong [:rstrong] (do not email) from comment #17)
> It doesn't change anything in the browser the user would typically ever see.
Good. That's what we want.

> It does change the update flow in that it will require the user to opt-in to
> the update if they have add-ons that are not compatible with the update 
I think what you're saying is that this change does what we want. The user would have to opt-in if they have add-ons that are not compatible. With add-ons default to compatible the user will not have to opt-in manually and should be silently updated. (This is what we're testing.) Is my understanding correct?

>and
> I think the update history which is available in preferences / options ->
> advanced -> update will display the version offered.
I don't think this is an issue.

Comment 19

[Robert Strong (they/them - no direct email)]

(In reply to Lawrence Mandel [:lmandel] from comment #18)
> (In reply to Robert Strong [:rstrong] (do not email) from comment #17)
> > It doesn't change anything in the browser the user would typically ever see.
> Good. That's what we want.
> 
> > It does change the update flow in that it will require the user to opt-in to
> > the update if they have add-ons that are not compatible with the update 
> I think what you're saying is that this change does what we want. The user
> would have to opt-in if they have add-ons that are not compatible. With
> add-ons default to compatible the user will not have to opt-in manually and
> should be silently updated. (This is what we're testing.) Is my
> understanding correct?
Yes

> >and
> > I think the update history which is available in preferences / options ->
> > advanced -> update will display the version offered.
> I don't think this is an issue.
Agreed

Comment 20

[Virgil Dicu [:virgil] [QA]]

(In reply to Lawrence Mandel [:lmandel] from comment #15)
> Virgil - Wanted to alert you of this change to assist in the testing of
> add-ons default to compatible.

Blair informed me previously of this while it was New. Thanks for bringing it in my attention now this is fixed. Will keep an eye on Beta 2 for this.

Comment 21

[Justin Wood (:Callek)]

Just for records:

SeaMonkey adopted this solution, transplanted to seamonkey-production in:
http://hg.mozilla.org/build/buildbotcustom/rev/7559067f5657
And added code to support SeaMonkey 2.7 in:
http://hg.mozilla.org/build/buildbotcustom/rev/954758bb10a4

Comment 22

[Rail Aliiev [:rail]]

FTR before I test, it looks like the patches prevent 10.0b1 -> 10.0b2 update:

<ashughes> rail: some more log info...
<ashughes> AUS:SVC Checker:onLoad - number of updates available: 1
<ashughes> AUS:SVC UpdateService:selectUpdate - skipping update because the update's application version is less than the current application version

https://aus3.mozilla.org/update/3/Firefox/10.0/20111221135037/WINNT_x86-msvc/en-US/releasetest/Windows_NT%205.1.3.0%20%28x86%29/default/default/update.xml?force=1

<?xml version="1.0"?>
<updates>
    <update type="minor" displayVersion="10.0 Beta 2" appVersion="10.0b2" platformVersion="10.0" buildID="20111228055358" detailsURL="https://www.mozilla.com/en-US/firefox/10.0/releasenotes/" actions="silent">
        <patch type="complete" URL="http://download.mozilla.org/?product=firefox-10.0b2-complete&amp;os=win&amp;lang=en-US&amp;force=1" hashFunction="SHA512" hashValue="7487b1cb117b312f98dd3ec231115a9c39f50f7522d062ba75162ecf7f3a27d7a85f289344dc709834c149670cde18a572d66449a6a2ae07a0f4278dde09e436" size="20326051"/>
        <patch type="partial" URL="http://download.mozilla.org/?product=firefox-10.0b2-partial-10.0b1&amp;os=win&amp;lang=en-US&amp;force=1" hashFunction="SHA512" hashValue="22c63746d60cf4de8605ec54905ed83d203c6ba90d031584a867e8e61ab3cd3d36277ac794cd840d30eb2fe3d891628941f47ef1b65722e8710ecc54aed2de01" size="1684606"/>
    </update>
</updates>

Comment 23

[Robert Strong (they/them - no direct email)]

As far as Firefox is concerned you are running 10.0 and 10.0 is greater than 10.0b2

Comment 24

[Rail Aliiev [:rail]]

Comment on attachment 583584 [details] [diff] [review]
run edit-snippets.sh for Firefox 10 betas

Backed out in http://hg.mozilla.org/build/buildbotcustom/rev/9082afaa6978 since it breaks updates...

Comment 25

[Justin Wood (:Callek)]

(In reply to Rail Aliiev [:rail] from comment #24)
> Comment on attachment 583584 [details] [diff] [review]
> run edit-snippets.sh for Firefox 10 betas
> 
> Backed out in http://hg.mozilla.org/build/buildbotcustom/rev/9082afaa6978
> since it breaks updates...

And backout out on seamonkey-production: http://hg.mozilla.org/build/buildbotcustom/rev/6d1cbf31b886

Comment 26

[[e-mail] (:bhearsum)]

I'll try to figure out a workaround this week.

Comment 27

[LegNeato]

In IRC we decided to just call it 11.0 in the AppVersion

Comment 28

[[e-mail] (:bhearsum)]

Attachment: use 11.0 for appVersion for 10.0 betas.

Comment 29

[u279076]

This seems to be fixed for 10.0b* -> 10.0b4 updates on releasetest.

Comment 30

[[e-mail] (:bhearsum)]

Great, thanks for confirming Anthony!