Sample assertion-centric test case and suite v2.0

VERIFIED INVALID

Status

VERIFIED INVALID
8 years ago
7 months ago

People

(Reporter: tsmith, Assigned: mozwebqa)

Tracking

Details

Attachments

(9 attachments)

(Reporter)

Description

8 years ago
Attached is a test case and test suite that merges Jeff's browserLoader example with our test case parameter landscape.  The essential attachments for discussion are the SmokeTestSuite, DemoPageLayout, DemoPageAddonLayout.  Other files are the usual supporting files with few or no changes.  Jeff's browserLoader sample from bug 558807 is included for reference since that bug will likely be closed.

Main features of the test case:
- Verifications are done with a set of small functions that test for a single condition and throw an assertion error when the condition isn't true.
- Test case executes only one value for each parameter (ex: one browser, one user type, one sort, etc).  Iterating through the values for each parameter is handled by the test suite.
- The list of add-ons is verified with a separate test case that takes add-on index# as a parameter.  Iteration through add-ons is driven by the test suite by using add-on index #s as an additional parameter list.

Main features of the test suite:
- Test suite gets the parameter lists used by a test case and constructs a suite that runs all test functions for a combination of parameter values.  Suites are created for each combination of values.  Running all parameter values for a test case could be done with one line of code, although the sample uses multiple lines for readability.
- The default smoke test parameters can be overridden as required for a test case.
(Reporter)

Comment 1

8 years ago
Created attachment 447135 [details]
smoke test suite with a test case implemented in two parts
Attachment #447135 - Flags: feedback?(vish.mozilla)
Attachment #447135 - Flags: feedback?(stephen.donner)
Attachment #447135 - Flags: feedback?(mozbugs.retornam)
Attachment #447135 - Flags: feedback?(krupa.raj82)
Attachment #447135 - Flags: feedback?(jbalogh)
(Reporter)

Comment 2

8 years ago
Created attachment 447136 [details]
test case for page elements except add-on list
Attachment #447136 - Flags: feedback?(vish.mozilla)
Attachment #447136 - Flags: feedback?(stephen.donner)
Attachment #447136 - Flags: feedback?(mozbugs.retornam)
Attachment #447136 - Flags: feedback?(krupa.raj82)
Attachment #447136 - Flags: feedback?(jbalogh)
(Reporter)

Comment 3

8 years ago
Created attachment 447137 [details]
test case for add-ons on the same page
Attachment #447137 - Flags: feedback?(vish.mozilla)
Attachment #447137 - Flags: feedback?(stephen.donner)
Attachment #447137 - Flags: feedback?(mozbugs.retornam)
Attachment #447137 - Flags: feedback?(krupa.raj82)
Attachment #447137 - Flags: feedback?(jbalogh)
(Reporter)

Comment 4

8 years ago
Created attachment 447138 [details]
Jeff's sample loader for browsers & other options
(Reporter)

Comment 5

8 years ago
Created attachment 447139 [details]
AMOlocators
(Reporter)

Comment 6

8 years ago
Created attachment 447141 [details]
AMOfunctions
(Reporter)

Comment 7

8 years ago
Created attachment 447143 [details]
general functions
(Reporter)

Comment 8

8 years ago
Created attachment 447144 [details]
TCparams - test case parameter values
(Reporter)

Comment 9

8 years ago
Created attachment 447145 [details]
gridConnection
(Reporter)

Updated

8 years ago
Assignee: nobody → mozwebqa
Attachment #447139 - Attachment mime type: text/x-python → text/plain
Attachment #447141 - Attachment mime type: text/x-python → text/plain
Attachment #447144 - Attachment mime type: text/x-python → text/plain
Attachment #447143 - Attachment mime type: text/x-python → text/plain
Attachment #447145 - Attachment mime type: text/x-python → text/plain
We can get rid of eval_test_case_result(testResult). There is a in-built python unittest function  
http://docs.python.org/library/unittest.html#unittest.TestResult.wasSuccessful
New code will look like:

test_result = runner.run(suite)

            if test_result.wasSuccessful():
                print 'all tests passed %s \n' %(test_result)  
            else:
                raise AssertionError(str(test_result))
Attachment #447135 - Flags: feedback?(vish.mozilla) → feedback-
Comment on attachment 447135 [details]
smoke test suite with a test case implemented in two parts

this has made the Suite extremely complex. If we want to run the test cases with different parameters and iterate through suite then the original method was serving that purpose. (Original method = one w/o init() in the test cases)
No longer valid with our new setup -> invalid.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → INVALID

Updated

7 years ago
Attachment #447135 - Flags: feedback?(jbalogh)

Updated

7 years ago
Attachment #447136 - Flags: feedback?(jbalogh)

Updated

7 years ago
Attachment #447137 - Flags: feedback?(jbalogh)
Status: RESOLVED → VERIFIED
Attachment #447135 - Flags: feedback?(stephen.donner)
Attachment #447135 - Flags: feedback?(mozbugs.retornam)
Attachment #447135 - Flags: feedback?(krupa.mozbugs)
Attachment #447136 - Flags: feedback?(vish.mozilla)
Attachment #447136 - Flags: feedback?(stephen.donner)
Attachment #447136 - Flags: feedback?(mozbugs.retornam)
Attachment #447136 - Flags: feedback?(krupa.mozbugs)
Attachment #447137 - Flags: feedback?(vish.mozilla)
Attachment #447137 - Flags: feedback?(stephen.donner)
Attachment #447137 - Flags: feedback?(mozbugs.retornam)
Attachment #447137 - Flags: feedback?(krupa.mozbugs)

Updated

7 months ago
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.