Closed
Bug 1058445
Opened 10 years ago
Closed 10 years ago
Switch to addons.allizom.org for add-on related remote tests
Categories
(Mozilla QA Graveyard :: Mozmill Tests, defect, P1)
Mozilla QA Graveyard
Mozmill Tests
Tracking
(firefox31 wontfix, firefox32 fixed, firefox33 fixed, firefox34 fixed, firefox-esr24 fixed, firefox-esr31 fixed)
People
(Reporter: whimboo, Assigned: whimboo)
References
Details
(Whiteboard: [mozmill-test-failure])
Attachments
(2 files)
3.74 KB,
patch
|
andrei
:
review+
whimboo
:
checkin+
|
Details | Diff | Splinter Review |
3.77 KB,
patch
|
andrei
:
review+
|
Details | Diff | Splinter Review |
Bug 899923 has shown that an unplanned outage of the addons-dev server can cause lots of test failures for add-on related Mozmill tests. This concerns me a lot especially when it comes to testing Firefox Beta and Release builds. A dev system totally unrelated to the Firefox product itself should absolutely not harm us that hard. So I propose that we change all the references to the AMO dev server to the production server. This will give way more stability. One thing I could consider is using an AMO staging server if such thing exists. Stephen or Krupa, could you give your feedback on this please? I know that WebQA wants the Mozmill test to be run against a pre-release of AMO, but e.g. the above implications make our tests unstable. So for testing against the dev server we might want to create a different testrun, which is not run for beta and release.
Flags: needinfo?(stephen.donner)
Flags: needinfo?(krupa.mozbugs)
Assignee | ||
Updated•10 years ago
|
Whiteboard: [mozmill-test-failure]
Comment 1•10 years ago
|
||
My only concern is that this will affect our user analytics data. How often do we run mozmill tests?
Flags: needinfo?(krupa.mozbugs)
Comment 2•10 years ago
|
||
(In reply to krupa raj[:krupa] from comment #1) > My only concern is that this will affect our user analytics data. How often > do we run mozmill tests? A gross estimate would be ~2800 testruns per week (850 daily + 2000 for beta / release). Where each testrun does hit AMO multiple times. Could we not blacklist the IP range to not be taken into consideration for the analytics data?
Assignee | ||
Comment 3•10 years ago
|
||
Krupa, you haven't replied to my other question yet, if there is a staging server available. Or is amo-dev the staging server?
Comment 4•10 years ago
|
||
(In reply to Henrik Skupin (:whimboo) from comment #3) > Krupa, you haven't replied to my other question yet, if there is a staging > server available. Or is amo-dev the staging server? As we discussed before the QA meeting, there is https://addons.allizom.org - Krupa can better-answer whether it's more stable/has the DB content you might need, Henrik.
Flags: needinfo?(stephen.donner)
Assignee | ||
Comment 5•10 years ago
|
||
Just to add... Reason why we had so many failures with add-on related tests against addons-dev might be that this instance gets updates each 15min, which also includes reloads of Apache. So sessions might be lost, or whatever else could happen. I assume that add-ons.allizom.org is not getting updated at the same time when we have Firefox releases?
Flags: needinfo?(krupa.mozbugs)
Comment 6•10 years ago
|
||
https://addons.allizom.org is probably more reliable than dev and i'm fine with using it.
Flags: needinfo?(krupa.mozbugs)
Assignee | ||
Comment 7•10 years ago
|
||
Wonderful. So lets make this change happen!
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Assignee | ||
Updated•10 years ago
|
Summary: Get rid of using http://addons-dev.allizom.org/ in add-on related tests → Switch to addons.allizom.org for add-on related remote tests
Assignee | ||
Comment 8•10 years ago
|
||
This patch changes the AMO domain we make use of. Further it removes the AMO site variable given that is nowhere used.
Attachment #8480178 -
Flags: review?(andrei.eftimie)
Assignee | ||
Updated•10 years ago
|
status-firefox31:
--- → wontfix
status-firefox32:
--- → affected
status-firefox33:
--- → affected
status-firefox34:
--- → affected
status-firefox-esr24:
--- → affected
status-firefox-esr31:
--- → affected
Comment 9•10 years ago
|
||
fyi, stage has code updates happen on wed, thur and friday.
Assignee | ||
Comment 10•10 years ago
|
||
Krupa, are there specific times set for those updates? Also how long do those usually take, involving downtime of the service?
Comment 11•10 years ago
|
||
Comment on attachment 8480178 [details] [diff] [review] amo-staging v1 Review of attachment 8480178 [details] [diff] [review]: ----------------------------------------------------------------- Looks good. All tests are passing. ::: lib/addons.js @@ +28,5 @@ > // AMO Preferences > const AMO_DISCOVER_URL = 'extensions.webservice.discoverURL'; > > +// AMO instance to use > +const AMO_DOMAIN = "addons.allizom.org"; I do wonder where we used AMO_PREVIEW_SITE. I remember it being necessary at some point, but now opening "addons.allizom.org" directly does get redirected to a workable URL, so we're good.
Attachment #8480178 -
Flags: review?(andrei.eftimie) → review+
Assignee | ||
Comment 12•10 years ago
|
||
(In reply to Andrei Eftimie from comment #11) > I do wonder where we used AMO_PREVIEW_SITE. I remember it being necessary at > some point, but now opening "addons.allizom.org" directly does get > redirected to a workable URL, so we're good. I think we only added that to have the full URL and wouldn't have to add the protocol each time. Landed on mozilla-central: https://hg.mozilla.org/qa/mozmill-tests/rev/f96b13c4b55d Lets observe todays tests and if all is fine lets get it backported asap.
Assignee | ||
Comment 13•10 years ago
|
||
Andrei tested my patch on beta and release today. All is fine on those branches. I did a quick spot-check for aurora and no failures are visible. So lets get the patch backported to all other branches. https://hg.mozilla.org/qa/mozmill-tests/rev/ef952fbbdb41 (aurora) https://hg.mozilla.org/qa/mozmill-tests/rev/00cf0f9ce9d1 (beta) https://hg.mozilla.org/qa/mozmill-tests/rev/1364a442facb (release) I will have to run some spotchecks for esr branches. So an update will follow soon. Btw. this will also drastically improve our stability for restart tests, which is a goal this quarter and tracked as bug 1035187.
Blocks: 1035187
Assignee | ||
Comment 14•10 years ago
|
||
Applies cleanly on esr31 and all tests pass: https://hg.mozilla.org/qa/mozmill-tests/rev/97ba5f074ba2 (esr31) Sadly there are merge conflicts for esr24. I will check shortly.
Comment 15•10 years ago
|
||
This should be the last ESR24 build, so lets not push this to esr24.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 16•10 years ago
|
||
No, there is still the chance for a chemspill. And seeing amo-dev down again would be a mess. This bug is important enough to get backported to esr24.
Assignee | ||
Comment 17•10 years ago
|
||
Backport patch for esr24.
Attachment #8480550 -
Flags: review?(andrei.eftimie)
Assignee | ||
Updated•10 years ago
|
Attachment #8480178 -
Flags: checkin+
Comment 18•10 years ago
|
||
Comment on attachment 8480550 [details] [diff] [review] amo_backport v1 Review of attachment 8480550 [details] [diff] [review]: ----------------------------------------------------------------- lgtm
Attachment #8480550 -
Flags: review?(andrei.eftimie) → review+
Assignee | ||
Comment 19•10 years ago
|
||
https://hg.mozilla.org/qa/mozmill-tests/rev/488cc02b7a85 (esr24)
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Updated•5 years ago
|
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•