Beta-N -> Beta fallback updates failing on releasetest for some locales

RESOLVED INCOMPLETE

Status

()

Core
Networking: HTTP
RESOLVED INCOMPLETE
7 years ago
2 years ago

People

(Reporter: ashughes, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(2 attachments)

(Reporter)

Description

7 years ago
Created attachment 560684 [details]
Update log

7.0b5 ja, ko, ru, zh-TW fallback updates on releasetest are failing to be found, but only under automation. I can fallback update these builds just fine if I test them manually.

I've attached a copy of the update log.
Something is wrong here by downloading the fallback patch:

*** AUS:UI gUpdates:onLoad - setting current page to startpage errorpatching
*** AUS:SVC Downloader:onStartRequest - original URI spec: http://download.mozilla.org/?product=firefox-7.0b6-complete&os=osx&lang=zh-TW&force=1, final URI spec: http://mozilla.3c.stonekitty.net/firefox/releases/7.0b6/update/mac/zh-TW/firefox-7.0b6.complete.mar
*** AUS:SVC Downloader:onStopRequest - original URI spec: http://download.mozilla.org/?product=firefox-7.0b6-complete&os=osx&lang=zh-TW&force=1, final URI spec: http://mozilla.3c.stonekitty.net/firefox/releases/7.0b6/update/mac/zh-TW/firefox-7.0b6.complete.mar, status: 2147549183
*** AUS:SVC Downloader:onStopRequest - non-verification failure
*** AUS:SVC getStatusTextFromCode - transfer error: 失敗 (不明原因), default code: 2152398849
*** AUS:SVC Downloader:onStopRequest - setting state to: download-failed
*** AUS:SVC Downloader:onStopRequest - all update patch downloads failed

This behavior only happens for a fallback of a partial patch. When I run the tests against 7.0b4 everything is fine. Same for 7.0b5 updates for en-US.

Rob, is there anything we could do to get more details about this unknown error? I could try to create a HTTP log tomorrow. But would be nice to hear if you have other ideas too. Thanks.
(In reply to Henrik Skupin (:whimboo) from comment #1)
> Rob, is there anything we could do to get more details about this unknown
> error? I could try to create a HTTP log tomorrow. But would be nice to hear
> if you have other ideas too. Thanks.

I should also CC Rob.
Created attachment 560986 [details]
HTTP log

So not sure what's going on here but we fail while downloading the complete mar file from the server. See the full log attached.

1880710336[100116260]: nsHttpChannel::OnStartRequest [this=100c73800 request=121af6680 status=0]
1880710336[100116260]: nsHttpChannel::ProcessResponse [this=100c73800 httpStatus=206]
1880710336[100116260]: nsHttpHandler::NotifyObservers [chan=c73850 event="http-on-examine-response"]
1880710336[100116260]: nsHttpChannelAuthProvider::CheckForSuperfluousAuth? [this=11a14e040 channel=100c73ad0]
1880710336[100116260]:   continuation state has been reset
1880710336[100116260]: nsHttpChannel::ProcessNormal [this=100c73800]
1880710336[100116260]:   calling mListener->OnStartRequest
1880710336[100116260]: nsHttpChannel::OnStopRequest [this=100c73800 request=121af6680 status=8000ffff]
1880710336[100116260]:   calling OnStopRequest
95973376[100140320]: nsHttpConnection::OnSocketReadable [this=688660]
95973376[100140320]: nsHttpConnection::CloseTransaction[this=688660 trans=22102720 reason=8000ffff]
95973376[100140320]: nsHttpTransaction::Close [this=22102720 reason=8000ffff]
95973376[100140320]: nsHttpConnectionMgr::ReclaimConnection [conn=688660]
95973376[100140320]: nsHttpTransaction::DeleteSelfOnConsumerThread [this=22102720]
95973376[100140320]: proxying delete to consumer thread...
95973376[100140320]: nsHttpConnection::Close [this=688660 reason=8000ffff]
95973376[100140320]: nsHttpConnectionMgr::OnMsgReclaimConnection [conn=100688660]
95973376[100140320]:   connection cannot be reused; closing connection
I don't think it's a fault in our automation scripts. It's probably triggered by the increased speed of the test. Moving over to Core:Network.
Component: Mozmill Automation → Networking: HTTP
Product: Mozilla QA → Core
QA Contact: mozmill-automation → networking.http
Steps to reproduce:

1. Get the Mozmill environment for your platform: https://mozqa.com/mozmill-env/
2. Execute setup.sh to install all dependencies
3. Clone the mozmill-automation repository: run.sh hg clone http://hg.mozilla.org/qa/mozmill-automation
4. Download the 7.0b5 zh-TW locale: run.sh download.py -v 7.0b5 -l zh-TW
5. Run the update test: run.sh testrun_update.py --channel=releasetest firefox-7.0b5.zh-TW.mac.dmg
Duplicate of this bug: 692626
(Reporter)

Comment 7

7 years ago
So it looks like this only happens for BetaN-1 to BetaN where N is the most current version in the candidate cycle.
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #7)
> So it looks like this only happens for BetaN-1 to BetaN where N is the most
> current version in the candidate cycle.

And only for releasetest. Could that be related to internal mirrors? Ben or Nick, does one of you have an idea?
(In reply to Henrik Skupin (:whimboo) from comment #8)
> (In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #7)
> > So it looks like this only happens for BetaN-1 to BetaN where N is the most
> > current version in the candidate cycle.
> 
> And only for releasetest. Could that be related to internal mirrors? Ben or
> Nick, does one of you have an idea?

Based on comment#1, I would bet that this isn't related to internal mirrors (mozilla.3c.stonekitty.net isn't one of them).

It's hard to say much more because this example happened with now-overwritten snippets. If you can repro again with the current snippets I can look at things in a bit more depth.
Ben, please check bug 692626 which Anthony has filed earlier today. It also happens for updates from 8.0b1 -> 8.0b2
Summary: 7.0b5 -> 7.0b6 fallback updates failing on releasetest for some locales → Beta-N -> Beta fallback updates failing on releasetest for some locales
(In reply to Henrik Skupin (:whimboo) from comment #10)
> Ben, please check bug 692626 which Anthony has filed earlier today. It also
> happens for updates from 8.0b1 -> 8.0b2

Hmmm, I don't see anything weird for the 8.0b1 -> 8.0b2 zh-TW windows update. I'm not sure what to tell you =\.
I was still able to reproduce it earlier today with 8.0b1 ru and the steps given in comment 5 from our machine in the QA lab.
Just tried it again on OS X 10.7 with 8.0b1 ru and it is still failing.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.