1186899 - Integrate the intern test suite with cloud Selenium hosting

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Code Cleanup, defect)

Description

[Luke Crouch [:groovecoder]]

To automatically run the intern test suite against stage & production servers after deployments (bug 1184670), we need to make a configuration for the Intern test runner to use Sauce Labs.

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

What's the status on this?  We can help with this, as you know!

Comment 2

[Matt Brandt [:mbrandt]]

In hindsight I posted to the wrong bug, however I generated encrypted saucelabs credentials (https://bugzilla.mozilla.org/show_bug.cgi?id=1197858#c1). David and I did get tests running using Travis+Intern+Sauce earlier this week. Although we have several errors to work through before the builds are stable.
- Running tests against the travis build (not dev)
- General suite errors of [current] unknown origin
- Tests not getting past the Persona Login workflow
- likely more, but we haven't compiled a full list yet.

Encrypted sauce credentials, note these use my Mozilla sub-account. Once we finish debugging and get the build green we may want to replace these with credentials linking to a different account.

 secure: "SnWizKd9zRohcjRO7Dpslz2VHLo3pYruz5p1KWcHDa3ttVxMXGhNz1jBUD/Lazn1uifqSoI8o8nMC503VXc+hu9Gq33wDjjoQRAU4srz13Acr6uRe7mNnGt3P9dKqODucMMqn0FvcZv+7z9vD73KNWLYk8n70Bd3+Wsmcre4kyE="
 secure: "fR75lloHnd0VT1pWuFy0Z4eExjb4/q+SPoC77xWiPXLmojsPUQYEZaGQCB4EPGioIHI4TWxM5dBkwyNrp+rOjJaWeZv3IIhpQshAtdbcckyET3Ang4CtJK/900SoS1ZyDag1kKg28vXYvJKyxcx6p0uUI+E1exTtCydjyLNp8rk="

Comment 3

[Luke Crouch [:groovecoder]]

:davidwalsh and I banged our heads on Intern + SauceLabs for a few hours today and had a hard time with it. In short, the Intern test runner seems have to have configuration bugs with its built-in SuaceLabsTunnel?

When we tried to run via BrowserStack it worked within 2 or 3 minutes.

We'll try updating Intern to 3.0 which may have easier SauceLabs config, but how likely is it we could use BrowserStack?

Comment 4

[Matt Brandt [:mbrandt]]

That's a good question for which I don't have enough knowledge to make committed comment on. Adding Stephend for a needsinfo.

Comment 5

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Luke Crouch [:groovecoder] from comment #3)

<snip>
 
> We'll try updating Intern to 3.0 which may have easier SauceLabs config, but
> how likely is it we could use BrowserStack?

We don't have any agreements in place with them - feel free and encouraged to contact them for their "Free for Open Source" plan, near the bottom of https://www.browserstack.com/pricing (if you're not already doing so).

Comment 6

[Luke Crouch [:groovecoder]]

Note: We're changing tack on this from using Travis and encrypted secrets to running Intern on our admin node with environment variables with Sauce Labs and/or BrowserStack credentials.

Comment 7

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/1133f42fd17b153c7f63d296de03454fb96a920c
fix bug 1186899 - documentation for cloud providers

https://github.com/mozilla/kuma/commit/55bcffbf4d1537dff989658c9a55e5b6c4bb63d2
Merge pull request #3531 from mozilla/intern-browserstack-on-deploy-1184670

fix bug 1186899 - documentation for cloud providers

Comment 8

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/aa329583c4112a0846ef46cc41afb278c326fc06
bug 1186899 - intern browserstack improvements

https://github.com/mozilla/kuma/commit/97a099e3976c39a7c3c9612241b6f4bf47a45a5a
Merge pull request #3547 from mozilla/intern-browserstack-2

bug 1186899 - intern browserstack improvements