Closed Bug 1522100 Opened 9 months ago Closed 8 months ago

Add Raptor support for running tests locally on the android-components reference browser

Categories

(Testing :: Raptor, enhancement, P1)

Version 3
enhancement

Tracking

(firefox67 fixed)

RESOLVED FIXED
mozilla67
Tracking Status
firefox67 --- fixed

People

(Reporter: rwood, Assigned: rwood)

References

Details

Attachments

(1 file, 1 obsolete file)

After further discussion with Christian:

  • Need the ability to pass in the 'activity' on the Raptor command line; currently it is only hard-coded for the geckoview_example app. Take that out and put it on the Raptor command line instead.

  • Christian suggested (and I agree) let's rename the Raptor command line app name from "geckoview" to "android".

i.e. The command line to run speedometer on the reference app should be:

./mach raptor-test --test raptor-speedometer --app=android --binary="org.mozilla.reference.browser" --activity=BrowserTestActivity

On second thought, I'm not going to use 'android' for the 'app' name - as we need to be able to differentiate between geckoview, reference browser, fennec, and future android apps - and I prefer not to have to rely on the 'binary' argument to check which app we are running on. I'll leave 'app=geckoview' for the geckoview example app, and add a new 'app=reference_browser" option for the reference browser.

(In reply to Robert Wood [:rwood] from comment #3)

Created attachment 9039173 [details]
Bug 1522100 - Add Raptor support for the android-components reference browser; r?davehunt

This patch adds support for running Raptor on the android components reference browser. This patch will just include the speedometer benchmark, although I have ran tp6m locally and it runs too - but there are issues with the page display (unsure if that is the reference browser or mitmproxy recordings) so will leave tp6m to future bug.

Note: Can only run this locally - unable to run this in production because the reference browser app is built in github and doesn't have treeherder/tc/try integration (see Bug 1522225).

This is the command to run speedometer on the reference browser:

 ./mach raptor-test --test raptor-speedometer --app=reference_browser --binary org.mozilla.reference.browser --activity=BrowserTestActivity

With the addition of the 'activity' command line arg, the Raptor test INI's for existing Raptor geckoview tests are also updated accordingly in this patch. This is the new command line to run speedometer on the geckoview_example app:

 ./mach raptor-test --test raptor-speedometer --app geckoview --binary org.mozilla.geckoview_example --activity=GeckoViewActivity
Summary: Add Raptor support for the android-components reference browser → Add Raptor support for running tests locally on the android-components reference browser
Blocks: 1524312

Depends on landing of the Raptor main class refactor in the first patch in Bug 1522972 (https://bugzilla.mozilla.org/show_bug.cgi?id=1522972#c11)

Depends on: 1522972

Note: Included in this patch, I'll also be adding Raptor command line support for Fenix - not support of any Raptor tests running on Fenix, but just command line support so the Fenix app will be able to be launched.

Attachment #9039173 - Attachment is obsolete: true

(In reply to Robert Wood [:rwood] from comment #7)

Created attachment 9045078 [details]
Bug 1522100 - Add Raptor support for running tests locally on the android-components reference browser; r?davehunt

Adds support for running raptor-speedometer, raptor-unity-webgl, and raptor-tp6m-1 pageload on the android Firefox reference browser on a locally connected android device.

Command line example:

./mach raptor-test --test raptor-speedometer --app=refbrow --binary org.mozilla.reference.browser --activity=BrowserTestActivity

There are now default 'activity' settings also, so you can just run this and it will use the default 'BrowserTestActivity' for the reference browser:

./mach raptor-test --test raptor-tp6m-amazon-refbrow --app=refbrow --binary org.mozilla.reference.browser

Also adds preliminary support for Fenix i.e. specifying 'app=fenix' on the command line; although actually running Raptor tests on Fenix has not been tested yet.

Note: Can only run this locally - unable to run this in production because the Reference Browser and Fenix apps are built in github and don't have treeherder/tc/try integration (see Bug 1522225).

Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/708d79591ac5
Add Raptor support for running tests locally on the android-components reference browser; r=davehunt
No longer depends on: 1522972
Status: ASSIGNED → RESOLVED
Closed: 8 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Blocks: 1536168
You need to log in before you can comment on or make changes to this bug.