538499 - incorrect instructions on the app update testing page

Status: RESOLVED FIXED

(quality.mozilla.org :: Content, defect)

Description

[Robert Strong (they/them - no direct email)]

http://quality.mozilla.org/documents-home/test-docs/update-testing

app.update.log.all has been changed to app.update.log as of Firefox 3.6

under Forcing Update Failure the update.status file must contain failed: 6 followed by a newline.

There was also a bug filed by QA where they modified the update.status and left
the updates directory open in explorer which will prevent the deletion of the
updates directory on some versions of Windows and thereby cause a failure.
Please add a note in the QA docs that after modifying the update.status the
update.status file must be closed and that the explorer window open to the
directory must also be closed.

I'm not sure where the QA docs for app update testing are but where ever they are they should have the following copied over from several comments in bug 538372 as well as the above.

Whether this is the cause or not there have been several bug reports filed by
QA where the update.status wasn't formatted properly. It has always required a
space between the : and the error number. For example, from Firefox 1.5
http://mxr.mozilla.org/mozilla1.8.0/source/toolkit/mozapps/update/src/nsUpdateService.js.in#1050

I looked at enforcing reasonable minimums and though it is easy to do so it
then makes the timer manager xpcshell tests take quite a bit longer. At least
for the time being I would prefer it if QA uses the following values when
testing app update.

The idle time must be less than the interval
user_pref("app.update.idletime", 1);

checks for updates every 30 seconds
user_pref("app.update.interval", 30);

timer fires every 30 seconds
user_pref("app.update.timer", 30000);

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

CC'ing Anthony, who has created the initial docs. Given the timer preferences those have only to be changed when we wanna test the notification but not when triggering the update manually, right?

Comment 2

[Robert Strong (they/them - no direct email)]

correct

Comment 3

[u279076]

I've made the suggested changes.  Please review http://quality.mozilla.org/documents-home/test-docs/update-testing and let me know if there are any changes you would like made.

Comment 4

[Aakash Desai [:aakashd]]

Verified with the changes. Thanks for getting this done, Anthony.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Thanks Anthony. I have one note:

For the Update Preferences section. You have written: "They are not required for manual update testing" which is not really explaining it correctly. Can you please update it to "They are not required when manually triggering the update via the Help menu".

Comment 6

[u279076]

(In reply to comment #5)
> Thanks Anthony. I have one note:
> 
> For the Update Preferences section. You have written: "They are not required
> for manual update testing" which is not really explaining it correctly. Can you
> please update it to "They are not required when manually triggering the update
> via the Help menu".

Done.

Comment 7

[Robert Strong (they/them - no direct email)]

I'm reviewing this as time permits and will post changes to the wording in individual posts. I'll let you know when I'm done in case you want to update it all at the same time


app.update.idletime

Set this to 1. It controls amount of idle time between update checks and must be less than app.update.interval.

should read:
Set this to 1. It controls amount of idle time before displaying the update prompt and must be less than app.update.interval.

Comment 8

[Robert Strong (they/them - no direct email)]

A couple of more

Update Types

There are four different types of updates which you will hear about while update testing.

Technically there are only two types of updates... minor and major.

major

Reword:
This type of update is typically from a previous major release of Firefox to the next major release of Firefox (e.g. Firefox 3.0.11 to Firefox 3.5.7).

New:
minor

This type of update is to a minor release of Firefox for the same major release (e.g. Firefox 3.0.11 to Firefox 3.0.12 or 3.5.1 to Firefox 3.5.5).


Complete and partial should be listed under Update Patch Type

New:
Update Patch Type

There are two types of update patches which you will hear about while update testing.

partial

Reword:
This type of update is only contains the changes between the old version and the new version of Firefox. It is smaller than the complete update. Partial patches are usually only created for minor updates between the current minor release and the previous minor release (e.g. Firefox 3.5.6 to Firefox 3.5.7).

complete

Reword:
This type of update contains all of the files for the new version of Firefox.  This update will be similar in size to the full download of Firefox. Complete patches are usually only used when updating to a new major release (e.g. Firefox 3.0.11 to Firefox 3.5.7) or when updating to a new minor version from a minor release that is not the previous minor release (e.g. Firefox 3.5.5 to Firefox 3.5.7).


fallback should probably just be under Forcing Update Failure.
fallback

This type of update is when a failure of an update is forced (see Forcing Update Failure below)

Comment 9

[Robert Strong (they/them - no direct email)]

If Update Preferences is added to this page (I thought they could be added / updated to an internal QA page) I'd prefer it to state "These preferences are only necessary if you are testing for the Update Notification." in the first sentence and if possible make the statement a tad stronger). It should also state strongly that these preferences should be reset immediately after the test is completed.

Comment 10

[Robert Strong (they/them - no direct email)]

5. Open update.status in a text editor and change "pending" to "failed: 6", make sure you insert a new line (press ENTER) after changing the value (see Locations of update.status below)

Should read:
5. Open update.status in a text editor and change "pending" to "failed: 6" (without the quotes), make sure there is one and only one new line (press ENTER) after the value (see Locations of update.status below)

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #9)
> If Update Preferences is added to this page (I thought they could be added /
> updated to an internal QA page) I'd prefer it to state "These preferences are
> only necessary if you are testing for the Update Notification." in the first
> sentence and if possible make the statement a tad stronger). It should also
> state strongly that these preferences should be reset immediately after the
> test is completed.

So we should probably only mention this settings in the accordingly existent Litmus test which tests this specific feature. Al is working on those tests so CC'ing him.

Comment 12

[u279076]

Suggested changes made.  Thanks for the feedback Rob, I've found it very useful.

As per comment 11, I've removed Update Preferences.  I've included the text below in case someone wishes to use it in a Litmus test...

---

Update Preferences
There are three update preferences which are recommended to be set in about:config prior to starting update testing. These preferences are only necessary if you are testing for the Update Notification.  They are not required if you are checking for updates manually via the Help menu.

app.update.idletime
Set this to 1. It controls amount of idle time between update checks and must be less than app.update.interval.

app.update.interval
Set this to 30. It tells Firefox to check for updates every 30 seconds.

app.update.timer
Set this to 30000.  It tells Firefox to fire a timer every 30 seconds.

To set these preferences, follow the steps below:
1. Type about:config into the location bar and press ENTER
2. When the warning page appears, click on I'll be careful, I promise!
3. In the Filter textbox, type app.update
4. Locate each of the above preferences and double click on them to change their values

Comment 13

[Robert Strong (they/them - no direct email)]

Thanks Anthony

Comment 14

[Robert Strong (they/them - no direct email)]

Filed bug 539308 for updating the litmus test