Closed Bug 1506912 Opened 6 years ago Closed 6 years ago

Raptor tp6 pageload on android geckoview

Categories

(Testing :: Raptor, enhancement, P1)

Product:
Testing
Component:
Raptor
Version:
Version 3
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rwood, Assigned: rwood)

References

Details

(Whiteboard: [geckoview:p1])

Attachments

(2 files, 1 obsolete file)

Bug 1506912 - Raptor support for tp6 pageload on android geckoview; r?jmaher,davehunt
47 bytes, text/x-phabricator-request
Details | Review
Bug 1506912 - Raptor tp6m-1 pageload test on android geckoview; r?jmaher,davehunt
47 bytes, text/x-phabricator-request
Details | Review 
Port raptor tp6 pageload to run on android geckoview (local android devices and in production @ bitbar). This will require:

- installing CA cert into geckoview
- turning on http/https proxy in geckoview
- recording mobile-specific mitmproxy pagesets
- playing back mitmproxy pagesets via mitmdump on the host but accessing the pages on geckoview on the mobile device via port fw
- taskcluster jobs/configs
Depends on: 1485082
Assignee: nobody → rwood
Status: NEW → ASSIGNED
Keywords: meta
Summary: [meta] Raptor tp6 pageload on android geckoview → Raptor tp6 pageload on android geckoview
Currently Raptor uses policies to import the mitmproxy certificate into Firefox Desktop. That method is not supported on android/geckoview (see https://bugzilla.mozilla.org/show_bug.cgi?id=1485082#c24).

:davidb (or other geckoview peeps) can you please tell me how I can programmatically import a security certificate into geckoview (or if that's possible?). Either that or if there's a way to bypass the security checking completely so the certificate isn't required (that's how we currently run pageload on google chrome).
Flags: needinfo?(dbolter)
No longer depends on: 1485082
James, to disable the sec in chrome Rob uses a command line argument, maybe that's a safe enough path forward for us here?
Flags: needinfo?(dbolter) → needinfo?(snorp)
We've kinda gone down this path before in bug 1197216. Dana did not seem open to adding any knobs for disabling cert checking back then.

Having some sort of CA management API in GeckoView seems like a fairly large task. Dana, is there an existing easy way to add a trusted CA that we can use here?
Flags: needinfo?(snorp) → needinfo?(dkeeler)
If you know what profile directory you'll be using, you can create an NSS certificate database with the appropriate trust settings beforehand (this is what we do for other automated tests that involve Firefox and TLS).

`certutil -N -d <path to profile> --empty-password` will create the necessary files (It should create "cert9.db", "key4.db", and "pkcs11.txt" in that directory. If it doesn't, prepend "sql:" to the profile path in these commands.)
`certutil -A -d <path to profile> -n "some nickname" -t TC,, -a -i <path to CA.pem>` will import and trust a root certificate to issue certificates for TLS server auth (the nickname doesn't really matter, but I think it needs to be unique)

The GeckoView API would need to call nsIX509CertDB.addCertFromBase64 with the contents of a PEM-encoded certificate (i.e. base64) but without the "-----BEGIN/END CERTIFICATE-----" header/footer and the trust string "TC,,".
Flags: needinfo?(dkeeler)
:rwood is the above usable for you?
Flags: needinfo?(rwood)
(In reply to Dana Keeler [:keeler] (she/her) (use needinfo) from comment #5)

Thanks Dana I'll see if I can get that to work with our automation. Question:

> The GeckoView API would need to call nsIX509CertDB.addCertFromBase64 with
> the contents of a PEM-encoded certificate (i.e. base64) but without the
> "-----BEGIN/END CERTIFICATE-----" header/footer and the trust string "TC,,".

Does that mean some work needs to be done in geckoview itself to support this method?
Flags: needinfo?(rwood)
(In reply to Geoff Brown [:gbrown] from comment #6)
> There's some mochitest code for certutil, if that's useful:
> 
> https://dxr.mozilla.org/mozilla-central/rev/
> c2593a3058afdfeaac5c990e18794ee8257afe99/testing/mochitest/runtests.py#1718

Thanks Geoff. Does certutil work on every platform? Where do you get the tool in automation, is it already installed on all the production machines? Or is it in the repo somewhere?
Flags: needinfo?(gbrown)
Flags: needinfo?(dkeeler)
certutil is included in the desktop builds' target.common.tests.tar.gz archives. 

For Android tests, where we need a linux certutil for the host, but we have an android build, certutil is included in the host-utils archive, downloaded from tooltool. There is a tooltool manifest at https://dxr.mozilla.org/mozilla-central/source/testing/config/tooltool-manifests/linux64/hostutils.manifest.
Flags: needinfo?(gbrown)
(In reply to Robert Wood [:rwood] from comment #8)
> Does that mean some work needs to be done in geckoview itself to support
> this method?

Yes - I was just saying that's what such an API would do if it existed.
Flags: needinfo?(dkeeler)
(In reply to Dana Keeler [:keeler] (she/her) (use needinfo) from comment #11)
> (In reply to Robert Wood [:rwood] from comment #8)
> > Does that mean some work needs to be done in geckoview itself to support
> > this method?
> 
> Yes - I was just saying that's what such an API would do if it existed.

Ah, ok I see, thank you!
Update: have the code working to create the cert nss db and import the mitmproxy cert, no errors (at least for me locally on OSX so far).

Next issue - looks like geckoview doesn't support turning on the browser proxy (http, ssl).

:snorp, do you know if the geckoview example app supports these prefs?

user_pref("network.proxy.http", "127.0.0.1")
user_pref("network.proxy.http_port", "8080")
user_pref("network.proxy.ssl", "127.0.0.1")
user_pref("network.proxy.ssl_port", "8080")

It looks like the proxy may not be working - when the URL is loaded in the geckoview example app, this error appears:

"BOOM! Sommething bad happened... ERROR_CATEGORY_URI: ERROR_UNKNOWN_HOST"
Flags: needinfo?(snorp)
I think those prefs should work fine. Are you forwarding port 8080 via adb?
Flags: needinfo?(snorp)
(In reply to James Willcox (:snorp) (jwillcox@mozilla.com) from comment #14)
> I think those prefs should work fine. Are you forwarding port 8080 via adb?

Yes, have tried both adb forward and adb reverse - because the mitmproxy server is running on the desktop host, but we want the android device to see it via the proxy. Pageload works great with live pages, but I can't seem to get the device to use mitmproxy on the host via the proxy. Still debugging more...
Ah, good news - I got it to work - adb reverse is fine - but I had the prefs wrong (quotes around the port) and some missing. Works with these:

user_pref("network.proxy.type", 1);
user_pref("network.proxy.http", "127.0.0.1");
user_pref("network.proxy.http_port", 8080);
user_pref("network.proxy.ssl", "127.0.0.1");
user_pref("network.proxy.ssl_port", 8080);
user_pref("network.proxy.no_proxies_on", "localhost, 127.0.0.1");

Loads the page into geckoview from the mitmproxy archive. The archive has the desktop site version, and the desktop page (google search) loaded entirely in geckoview. Next step is to try recording a mobile version of the page with mitmproxy and use that.
Attachment #9027250 - Attachment is obsolete: true
(In reply to Robert Wood [:rwood] from comment #17)
> Created attachment 9030544 [details]
> Bug 1506912 - Raptor support for tp6 pageload on android geckoview; r?jmaher,davehunt

This patch adds support to Raptor to run tp6 pageload tests on android geckoview (geckoview example app). This patch doesn't add the actual test and job configs. This patch can be landed first. - I will use follow-up patches to add the mobile recording(s) and create the actual test jobs etc. If you wish to test this patch locally:

1. Add the following test section to the raptor-tp6-1.ini [1]:

[raptor-tp6-google-geckoview]
page_cycles = 25
apps = geckoview
test_url = https://www.google.com/search?hl=en&q=barack+obama&cad=h
playback_recordings = google-search.mp
measure = fnbpaint, hero, dcf, ttfi
hero = hero1

2. Then with the geckoview example app loaded on your device, invoke Raptor with:

./mach raptor-test --test raptor-tp6-google-geckoview --app=geckoview --binary="org.mozilla.geckoview_example"

[1] https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-1.ini
Keywords: leave-open
Whiteboard: [geckoview:p1]
Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/790e6f59a355
Raptor support for tp6 pageload on android geckoview; r=jmaher
Next patch will be the actual Raptor test and taskcluster configs for 'raptor-tp6m-1', which will test pageload on geckoview for:

https://www.amazon.com
https://www.facebook.com
https://www.google.com
https://www.youtube.com

I have made the mitmproxy recordings for the above sites on a Google Pixel 2 using the geckoview example app. I have perms to upload to tooltool now, so I should be able to upload the recordings archive myself.
Depends on: 1514391
"21:44:47     INFO -  raptor-mitmproxy unable to find certutil at /builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/certutil"

Hey :bc, can you pls verify that I have the correct path for the certutil binary inside the bitbar/tc image? I still can't seem to find it. Thanks!
Flags: needinfo?(bob)
With the recent update to host-utils, it is now at /builds/worker/workspace/build/hostutils/host-utils-66.0a1.en-US.linux-x86_64. That might be backed out though in Bug 1516441.
Flags: needinfo?(bob)
Priority: -- → P1
Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ef3912c3403b
Raptor tp6m-1 pageload test on android geckoview; r=jmaher
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Keywords: leave-open
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: