1209719 - [test-manifests] create a demo product in services-test w/ corresponding test manifest(s)

Status: VERIFIED FIXED

(Cloud Services :: QA: Test Automation, defect)

Description

[Richard Pappalardo [:rpapa][:rpappalardo]]

SUMMARY

So that each test can be executed identically (run.sh), each test will need a clean way to grab all the arguments it needs from services-test.

Currently the test manifest.json is specified on a per-product basis and can be found here:
https://github.com/mozilla-services/services-test/<app_name>/manifest.json

Although the current format lists test-types and environments, it does not have a mechanism for providing any additional parameters such as URLs, etc.  

NOTES:
[1]. if we provide all such arguments in one manifest file, it may become unwieldly.
[2]. we could split off additional arguments into per-environment child manifests as needed:  stage_e2e-test.json, stage_loadtest.json, prod_e2e-test.json, etc.

Comment 1

[Peter deHaan [:pdehaan]]

Example formatting/layout. Keeps the manifests nicely hidden in a folder.

```
pocket/
  manifest.json
  manifests/
    stage_stack-check.json
    pre-prod_e2e-test.json
    prod_urlcheck.json
    stage_security.json


{
  envs: {
    stage: {
      test-types: ["..."]
    },
    pro-prod: {
      test-types: ["..."]
    }
  }
}
```

Comment 2

[Richard Pappalardo [:rpapa][:rpappalardo]]

Adding demo project:
https://github.com/mozilla-services/services-test/tree/bug/1209719/demo

Comment 3

[Richard Pappalardo [:rpapa][:rpappalardo]]

from :Silne30 - mozbase has a tool called manifestparser we should consider for structuring our test manifests:
http://mozbase.readthedocs.org/en/latest/manifestparser.html#manifestparser-create-and-manage-test-manifests

Comment 4

[anju jacob]

[Tracking Requested - why for this release]: