780955 - Support writing HTML tests for security checks

Status: RESOLVED INCOMPLETE

(Core :: DOM: Device Interfaces, defect, P1)

Description

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

There are two kinds of tests I want to write
 - pure-HTML tests in which apps poke APIs they don't have permissions to access
 - C++ tests in which specially-crafted code pokes IPDL interfaces it shouldn't

For the HTML tests,

cjones> mounir, fabrice, hey, we're going to need tests for the permissions stuff ... what's the state of the art for setting up apps for testing?
<mounir> cjones: two solutions: you have an HTML chrome test and you insert <iframe mozapp>
<mounir> or you have a mochitest and you add it in the whitelist
* DanielColoma has quit (Quit: http://www.mibbit.com ajax IRC Client)
<mounir> cjones: allowed apps manifest urls are in build/automation.py.in (around line 525)
<cjones> mounir, so with the chrome test i can forge a manifest?
<cjones> and bypass the static list?
* mariow ([e-mail]) has joined #webapi
* sicking has quit (Ping timeout)
* victorporof has quit (Ping timeout)
* sicking ([e-mail]) has joined #webapi
<mounir> cjones: hmm, you can probably use mozapp.install()

Not blocking the permissions infra on this because we need to get that landed and integrated into patches asap.

Comment 1

[Andreas Gal :gal]

Definitely a want, but I wouldn't block a release on this.

Comment 2

[Lucas Adamski [:ladamski]]

Seems that we need to be able to do testing of permissions.  This is currently the source of much pain and several email threads.  Unless I misunderstand this bug.

Comment 3

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

One issue per bug please.

Morphing this to just cover the first issue mentioned in comment 0. I.e. writing HTML tests which check that we prompt and block requests as needed.

Please file a separate bug on the C++ thing.

Comment 4

[Mounir Lamouri (:mounir)]

(In reply to Jonas Sicking (:sicking) from comment #3)
> One issue per bug please.
> 
> Morphing this to just cover the first issue mentioned in comment 0. I.e.
> writing HTML tests which check that we prompt and block requests as needed.
> 
> Please file a separate bug on the C++ thing.

Testing access to an API with permission enabled/disabled should be easy as soon as permission manager is app-aware.
Testing if the prompts shows up can be done by wrinting a mock of nsIPermissionPrompt (or whatever this is called). Just as we do for nsIFilePicker.

Comment 5

[Andrew Overholt [:overholt]]

Mochi tests require 2 infrastructure pieces:

1. Being able to test if a dialog comes up.
2. Verifying that a mochi test running a trusted or certified app can do what they are allowed.

Comment 6

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

LOE'ing at Medium given current knowledge, but think it's towards the 1 week side of it and I was on the fence for Small.

We have some rough code to do the mock prompt and poke APIs from an app context (the intent of the HTML check portion). We don't yet have a fully working general solution for all privileged and certified app possibilities, and may not have identified all the blockers.

This is a bit of an odd bug overall, though, as we don't expect to require product code change to make this work. It will be part of the permissions test code in general.

Comment 7

[Andrew Overholt [:overholt]]

How's this going, Geo?

Comment 8

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Basic rollup of current status is here: https://etherpad.mozilla.org/ep/pad/view/permtesting-20121003/KGDaYi2Tkq

Based on dependencies listed here, added a couple of bugs to Depends On.

Comment 9

[Andrew Overholt [:overholt]]

Are we waiting on automation?  Can we get going on this bug without it?

Comment 10

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

AFAIK, this bug refers directly to the automation. 

We're going to discuss this week whether anything can be partially automated to start some testing on local computer (as opposed to targeting CI) but any significant testing--particularly on B2G--likely does require the automation framework to be up and running.

Comment 11

[Alex Keybl [:akeybl]]

Milestoning for C2 (deadline of 12/10), as this meets the criteria of "known bugs with LOE:M".

Comment 12

[Dietrich Ayala (:dietrich)]

No update since October. What's the status?

And does this truly need to block the release?

Comment 13

[Lawrence Mandel [:lmandel] (use needinfo)]

There's been no action here for some time now. We're not going to block on new test infrastructure at this point in the schedule.

Comment 14

[David Chan [:dchan]]

(In reply to Lawrence Mandel [:lmandel] from comment #13)
> There's been no action here for some time now. We're not going to block on
> new test infrastructure at this point in the schedule.

I don't believe we need anymore test infrastructure changes for this bug. However the bug has be repurposed to keep track of test writing status. There has been progress on those dependent bugs.

Comment 15

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

FxOS no longer in tree. Marking old FxOS Device Interfaces bugs as INCOMPLETE.