Closed Bug 1585046 Opened 1 year ago Closed 1 year ago

[meta] Expand the raptor-browsertime unit test coverage

Categories

(Testing :: Raptor, enhancement, P2)

Version 3
enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rwood, Assigned: onegru)

References

(Blocks 1 open bug)

Details

(Keywords: meta)

Add more raptor framework python unit tests. Note: Focus on the browsertime-raptor specific code path, since we may be moving to browsertime soon and eventually removing the raptor webext, control server etc.

Blocks: 1582756

Octavian can you take this one please

Assignee: nobody → onegru
Status: NEW → ASSIGNED
Flags: needinfo?(onegru)

Yes, I will work on it after finishing work on Bug 1585966.

Flags: needinfo?(onegru)

Unit testing can be placed in testing/raptor/test/test_raptor.py file,
since both Raptor and Browsertime components reside in testing/raptor/raptor/raptor.py file.

The unit testing of the Browsertime component can be done with the following steps:
1. Adapt Browsertime to already-written test-cases (add Browsertime to @pytest.mark.parametrize):
-test_build_profile
-test_start_browser
2. Write new test case for run_test method.

Proposal: write tests only for public Browsertime methods. The methods in which Browsertime have its inner logic, like
-remove_mozprofile_delimiters_from_profile
-check_for_crashes
-clean_up
-run_test_setup
-run_test_teardown
must not be tested, because they will make the test cases more brittle.

Note: step 1 is already done and will be submitted along with step 2 in one single commit.

Edit: this comment is outdated. The discussion related to raptor.py tested components will continue here

I see that Browsertime's integration is mostly spread across 3 main modules:

We should split this ticket in 3 subtasks, one for each module.

Depends on: 1590035
Keywords: meta
Depends on: 1590038
Depends on: 1590040
Summary: Expand the raptor-browsertime unit test coverage → [meta] Expand the raptor-browsertime unit test coverage

note:
keep in mind that output.py and results.py will be refactored in the future so maybe we should not spend to much time in these modules.

OK, I will try not to refactor, but to better shape the code for easy testing.
No breaking changes will be added.

Keywords: leave-open

Hey Octavian, are you finished with this bug or are you adding more unit tests? If you're finished please close, thanks!

Flags: needinfo?(onegru)

Finished, resolved. Thanks for reminding :)

Flags: needinfo?(onegru)
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.