Closed Bug 846226 Opened 7 years ago Closed 3 years ago

Create a "hominid" tool for more intelligent UI state-space exploration

Categories

(Core :: General, defect)

x86_64
Linux
defect
Not set

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: cjones, Unassigned)

References

Details

We have monkey and orangutan tools to send random input events to code under test.  That's invaluable fuzzing and we should keep running it.

In the same vein, we can do more precise UI fuzzing by randomly exploring an abstracted model of the UI.  Instead of sending taps randomly to find buttons, if the UI sends us a model that defines where the clickable elements are, we can just directly tap them to transition into a new UI state.

This kind of testing would let us find higher-level bugs more efficiently, but it's complementary to monkey testing because it relies on models provided by content, which can themselves be buggy.  Monkey testing doesn't care what's on screen so isn't misdirected by those kinds of bugs.
Summary: Create a "hominid" tool for UI state-space exploration → Create a "hominid" tool for more intelligent UI state-space exploration
Rwood and I were talking about a version of something like this today. He's written several long running tests that do various things for as many iterations as you like. He was thinking that instead of monkey you simply shuffle these tests for small numbers of iterations over a long time sequence.

This gets us much closer to automating something akin to user interaction and if used with a seed, we have a way to make these completely reproducible sequences of actions.

For instance by chaining his tests together like this you could:
* add 3 contacts
* have sms conversation for 10 messages
* add 5 calendar events
* go into airplane mode, come out
* watch a video
* delete 2 contacts
* etc
in a loop for as many iterations as you want.  It has some disadvantages that we are paving cow paths, and so we're not going to find as many wonky things as a more pure fuzzing approach gets us, but it has some benefits too in that we are running actions that people do everyday for long periods of time to get some idea of real world stability.

Perhaps we want Rob's tool and this other hominid tool. Perhaps rob's tool and generic fuzzing give us enough coverage that we're good here.  I'm not totally sure it solves the problem you have in mind Chris, but it's certainly a step in the right direction and we'll have it rather soon.
Whiteboard: [platform-rel-Intel]
platform-rel: --- → ?
platform-rel: ? → ---
Whiteboard: [platform-rel-Intel]
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.