1082861 - Update test didn't fail after 404 for partial

Status: RESOLVED INVALID

(Mozilla QA Graveyard :: Mozmill Tests, defect, P1)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: console log

For Firefox 31.2.0esr there was an issue on the RelEng side where download.m.o had not been set up for the partial update from 24.8.1, so 404 responses were returned. The mozmill tests didn't mark the test as failed, just failed over to the complete:

Short part of log (see attachment for full log, or
http://mm-ci-master.qa.scl3.mozilla.com:8080/job/ondemand_update/96780/):

12:42:43 *** AUS:SVC Downloader:onStopRequest - original URI spec: http://download.mozilla.org/?product=firefox-31.2.0esr-partial-24.8.1esr&os=osx&lang=kk&force=1, final URI spec: http://download.mozilla.org/?product=firefox-31.2.0esr-partial-24.8.1esr&os=osx&lang=kk&force=1, status: 2147549183
12:42:43 *** AUS:SVC Downloader:onStopRequest - status: 2147549183, current fail: 0, max fail: 10, retryTimeout: 2000
12:42:43 *** AUS:SVC Downloader:onStopRequest - non-verification failure
12:42:43 *** AUS:SVC getStatusTextFromCode - transfer error: Сәтсіз (себебі белгісіз), default code: 2152398849
12:42:43 *** AUS:SVC Downloader:onStopRequest - setting state to: download-failed
12:42:43 *** AUS:SVC Downloader:onStopRequest - verification of patch failed, downloading complete update patch

I was surprised by this, as I recall QA catching this issue in the past. Is it working as you expect ?

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I do not see this as a fault on our side. At least not as a regression. Actually we would have caught this failure, if no download of the full patch has happened immediately but the error page would have been shown.

Also the test code as we have so far does not check if the downloaded file size is the same as what the update snippet says. We only retrieve the type of the patch from the update service. Given that we haven't noticed the problem we might wanna improve our tests.

But anyway there might have been a change for OS X by the app bundle refactoring Stephen recently worked on. Maybe he or Rob have an answer to it, or what to best check for.

Comment 2

[Robert Strong (they/them - no direct email)]

It likely immediately went for the complete as the client should do since in this case not informing the user and downloading the complete would be the right thing to do. Also, I did most of the app update work for V2 signing and the only thing that changed in regards to this was the download location.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Ok, so the question then is if it is helpful to test this. I would say yes, because it's a failure we should detect. Even the user is not seeing this, it would mean a larger download size. For us we would have to multiply by users, which is kinda lot of extra traffic.

Nick, if you want to see this tested, we can certainly add something into our update tests.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

I'd like to see a test added.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Ok, so this is indeed a kinda important test to have. Our tests should detect if the updates as given by the update snippet are available and downloaded, or not.

While we are getting this implemented we can also start to store the update snippet data in our results for later investigation. There is already bug 640524 filed for that, which will fix the first half part.

Andreea, I think that we should take care of it soonish. Maybe one of you has a bit of time between all the other work?

Comment 6

[Andreea Matei [:AndreeaMatei]]

Yes, Andrei has some hours left for other tasks that might need our attention.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Andreea Matei [:AndreeaMatei] from comment #6)
> Yes, Andrei has some hours left for other tasks that might need our
> attention.

Just to add, he would have to work on bug 640524 first. So adding the other one first to the next sprint might make more sense. But thanks for taking care of it!

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozmill tests have been superseded by Marionette tests.