Closed Bug 994962 Opened 10 years ago Closed 9 years ago

run loop client shared & non-shared unit tests against Chrome, both by hand and in CI

Categories

(Hello (Loop) :: Client, defect, P3)

Product:
Hello (Loop)
Component:
Client
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Blocking Flags:
backlog tech-debt

People

(Reporter: dmosedale, Assigned: standard8)

References

Details

(Whiteboard: [tech-debt][fixed by bug 1232441])

User Story

Suggested Initial version for this bug:

- Run unit tests on travis CI from loop-client per checkin to master
- Use selenium harness to run Chrome (prob extract from Talkilla)
- Add irc hook, and email various people for Travis failures

Possible follow-up:

- Integrate into tbpl when Marionette supports WebDriver/Chrome more fully
- Set up a branch on loop-client for fx-team and auto-update it, and run tests straight away.
This will involve either porting frontend_tester.py from Marionette to WebDriver, or porting test_frontend_all.py from Talkilla.  If the interoperability is good enough, it's conceivably that the Marionette tests could be run against the Chrome WebDriver backend, but that could be fragile.

In some ideal world, we'd be able to put these test results into Tbpl.  Exactly how practical / how much work that would be is not clear to me, but perhaps the ateam folks have some thoughts here...
If the tests are stable and don't have external network dependencies, it shouldn't be too hard to get them running in TBPL.  We can certainly help with that piece.
Dan - its unclear to me what the intention is for non-shared unit tests for standalone. Is that this bug or some other? I think we do want to resolve those here.

My original thoughts were:

- Shared & Desktop tests run in FF via TBPL
- Shared & Standalone tests run in other apps via some sort of CI system - especially given the standalone ones will likely be in a separate repo.

There's a slight non-coverage here in the Standalone tests being run in Firefox, but I'm not convinced we need full coverage on all of these, especially given what we're sharing (this may change when we get to functional/integration tests).
Flags: needinfo?(dmose)
The non-shared standalone tests had actually slipped my mind. I was actually kinda running with the idea that we just run all the tests in Tinderbox to avoid having multiple different CI systems that we had to interact with. That said, there are almost certain to be advantages to doing it the way you suggest as well.

This bug can certainly cover the Chrome case and stay as an MVP thing.

I think we want a separate bug for running the client-local tests for MLP, however.  I've filed bug 995355 for that; it should be trivial.
Flags: needinfo?(dmose)
Summary: run loop client shared unit tests against Chrome, both by hand and in CI → run loop client shared & non-shared unit tests against Chrome, both by hand and in CI
Priority: -- → P3
Whiteboard: [s=fx33, p=.25]
Target Milestone: --- → mozilla33
Priority: P3 → P1
Whiteboard: [s=fx33, p=.25] → --do_not_change-- [mozilla33 carry over]
Target Milestone: mozilla33 → mozilla34
Whiteboard: --do_not_change-- [mozilla33 carry over] → [mozilla33 carry over]
Now that we've got loop-client tests separate, I think we could consider running the Chrome tests in travis.

I'm just going to xref bug 1020675 and https://github.com/mozilla/loop-client/pull/35 as possible things we might want to look at when doing this.
Whiteboard: [mozilla33 carry over] → [mozilla33 carry over][p=1, spike]
I'd love to have this, but I think we can push this to the Fx35 cycle
Whiteboard: [mozilla33 carry over][p=1, spike] → [mozilla34 carry over][p=1, spike]
Target Milestone: mozilla34 → mozilla35
Whiteboard: [mozilla34 carry over][p=1, spike] → [mozilla34 carry over][p=1, spike][tech-debt]
backlog: --- → Fx36+
backlog: Fx36+ → Fx37+
Target Milestone: mozilla35 → ---
Priority: P1 → P2
backlog: Fx37+ → Fx38+
Priority: P2 → P3
Whiteboard: [mozilla34 carry over][p=1, spike][tech-debt] → [p=1, spike][tech-debt]
backlog: Fx38+ → tech-debt
User Story: (updated)
Whiteboard: [p=1, spike][tech-debt] → [tech-debt]
Blocks: 1230204
Rank: 35
This has been fixed by bug 1232441
Assignee: nobody → standard8
Depends on: 1232441
Whiteboard: [tech-debt] → [tech-debt][fixed by bug 1232441]
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.