Open Bug 597560 Opened 10 years ago Updated 4 years ago

Firefox 4 Beta Study v2: Tracking Bug

Categories

(Mozilla Labs Graveyard :: Test Pilot Studies, defect)

defect
Not set
blocker

Tracking

(Not tracked)

People

(Reporter: jono, Unassigned)

Details

(Whiteboard: FF4beta Interface 2 )

Attachments

(1 file, 2 obsolete files)

We are releasing a new version of the Firefox 4 beta study.  The study code is in the hg.mozilla.org/labs/testpilotweb repo, in the file testcases/beta/combined.js

On the web: http://hg.mozilla.org/labs/testpilotweb/raw-file/066aeec111f5/testcases/beta/combined.js

Highest priority features to be added to this version, over the previous version, are:

 * Sync instrumentation: How many people have configured sync, and how recently did they sync up?
 * Panorama instrumentation: Observing when people click the sync button or hit the keyboard shortcut; recording how many tab groups they have and how many tabs per group.
 * App tab instrumentatin: Observing when someone makes a tab into an app tab or back, and counting how many app tabs people have.

Secondary priorities are the items in red on the tracking spreadsheet at https://spreadsheets.google.com/ccc?key=0AnJReXqM7IBPdGNnM0xyOGVJXzFydS1ScERGNUJvU1E&hl=en#gid=0

Things for testers and code reviewers to watch out for:

- Are all events recorded correctly on every operating system?

- Are we throwing any exceptions for conditions that should be caught and handled more gracefully - for example, attempting to register listeners on interface elements that don't happen to exist on the current OS or build

- Have we kept up with all the changes to the Firefox 4 interface in the last couple of betas - do all our observers that used to work, still work?

- Do we record UI customizations correctly, and do the observers still work right no matter how someone has customized their UI?

- Are we correctly recording the difference between using something by clicking the toolbar, clicking the menu, hitting the keyboard shortcut, and using a context menu?

- Are there any ways we haven't considered yet to get the "missing" UI interactions - for example, the keyboard shortcuts for cut/copy/paste, which do not appear to generate any command or key events on the main command set or main menu bar.
More things for testers and code reviewers to look at:

All users with any of the 4.0 betas should be getting and running this test, whether or not they ran the Beta Interface Study round 1.
(Round 1 had id = 100; round 2 has id = 101.)

If a user submit both round 1 and round 2, they should have the same GUID attached to both submissions, for longitudinal comparison purposes.  (This works by having the round 2 study look at the GUID stored in prefs for round 1, and copying this over its own GUID).
Attached file The code for the study itself. (obsolete) —
Attachment #478120 - Flags: review?(avarma)
Attached file The code for the beta interface study (obsolete) —
Updated version of the study code itself to match up with latest revision in Hg.
Attachment #478120 - Attachment is obsolete: true
Attachment #478120 - Flags: review?(avarma)
Comment on attachment 479101 [details]
The code for the beta interface study

I assume I am reviewing this, so setting r? to me.
Attachment #479101 - Flags: review?(avarma)
Comment on attachment 479101 [details]
The code for the beta interface study

My review comments are here:

http://github.com/toolness/my-code-reviews/commit/83f34cf9b5c3a0aaa61f5840a4e11f29c7e4fcc2

Feel free to reply to the comments on that page.
Attachment #479101 - Flags: review?(avarma) → review-
Some changes to the code made in http://hg.mozilla.org/labs/testpilotweb/rev/58f97868d104  and http://hg.mozilla.org/labs/testpilotweb/rev/b70add898bda

1. now recording tab groupings when user closes panorama interface
2. now correctly recording sub-menu items of windows appmenu button
Attached the latest version of the study code with the latest bug fixes.
Attachment #479101 - Attachment is obsolete: true
Attachment #481533 - Flags: review?(avarma)
Comment on attachment 481533 [details]
The code for the study itself.

All my review comments so far are here:

http://github.com/toolness/my-code-reviews/pull/1
Attachment #481533 - Flags: review?(avarma) → review-
Whiteboard: FF4beta Interface 2
Product: Mozilla Labs → Mozilla Labs Graveyard
You need to log in before you can comment on or make changes to this bug.