567819 - Sample assertion-centric test case and suite v2.0

Status: VERIFIED INVALID

(Testing Graveyard :: WebQA, defect)

Description

[Truman Smith [:truman]]

Attached is a test case and test suite that merges Jeff's browserLoader example with our test case parameter landscape.  The essential attachments for discussion are the SmokeTestSuite, DemoPageLayout, DemoPageAddonLayout.  Other files are the usual supporting files with few or no changes.  Jeff's browserLoader sample from bug 558807 is included for reference since that bug will likely be closed.

Main features of the test case:
- Verifications are done with a set of small functions that test for a single condition and throw an assertion error when the condition isn't true.
- Test case executes only one value for each parameter (ex: one browser, one user type, one sort, etc).  Iterating through the values for each parameter is handled by the test suite.
- The list of add-ons is verified with a separate test case that takes add-on index# as a parameter.  Iteration through add-ons is driven by the test suite by using add-on index #s as an additional parameter list.

Main features of the test suite:
- Test suite gets the parameter lists used by a test case and constructs a suite that runs all test functions for a combination of parameter values.  Suites are created for each combination of values.  Running all parameter values for a test case could be done with one line of code, although the sample uses multiple lines for readability.
- The default smoke test parameters can be overridden as required for a test case.

Comment 1

[Truman Smith [:truman]]

Attachment: smoke test suite with a test case implemented in two parts

Comment 2

[Truman Smith [:truman]]

Attachment: test case for page elements except add-on list

Comment 3

[Truman Smith [:truman]]

Attachment: test case for add-ons on the same page

Comment 4

[Truman Smith [:truman]]

Attachment: Jeff's sample loader for browsers & other options

Comment 5

[Truman Smith [:truman]]

Attachment: AMOlocators

Comment 6

[Truman Smith [:truman]]

Attachment: AMOfunctions

Comment 7

[Truman Smith [:truman]]

Attachment: general functions

Comment 8

[Truman Smith [:truman]]

Attachment: TCparams - test case parameter values

Comment 9

[Truman Smith [:truman]]

Attachment: gridConnection

Comment 10

[Vishal Kamdar [:vish_moz]]

We can get rid of eval_test_case_result(testResult). There is a in-built python unittest function  
http://docs.python.org/library/unittest.html#unittest.TestResult.wasSuccessful
New code will look like:

test_result = runner.run(suite)

            if test_result.wasSuccessful():
                print 'all tests passed %s \n' %(test_result)  
            else:
                raise AssertionError(str(test_result))

Comment 11

[Vishal Kamdar [:vish_moz]]

Comment on attachment 447135 [details]
smoke test suite with a test case implemented in two parts

this has made the Suite extremely complex. If we want to run the test cases with different parameters and iterate through suite then the original method was serving that purpose. (Original method = one w/o init() in the test cases)

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

No longer valid with our new setup -> invalid.