Closed Bug 1181822 Opened 7 years ago Closed 6 years ago

Add an A/B test to for low-res tiles vs. fading tiles

Categories

(Firefox for Android Graveyard :: Toolbar, defect)

34 Branch
defect
Not set
normal

Tracking

(firefox49 fixed)

RESOLVED FIXED
Firefox 49
Tracking Status
firefox49 --- fixed

People

(Reporter: snorp, Assigned: snorp)

References

Details

Attachments

(3 files, 1 obsolete file)

We could try only doing this in Nightly to start with. The reasoning here is:

1) We have a lot of issues with low-res tiles appearing when they shouldn't. The gfx folks are looking into this stuff now, but...

2) It seems like it may be more trouble than it's worth.

3) With bug 1178376 applied, which allows fading in new tiles, checkerboarding feels a little less jarring.

I propose we try the fading tiles stuff on Nightly to see how folks like it.
Attachment #8631306 - Flags: review?(mark.finkle)
Attachment #8631306 - Flags: review?(blassey.bugs)
Attachment #8631306 - Flags: feedback?(alam)
Gotta screenshot? :D
Flags: needinfo?(snorp)
I suspect you'll want a video or a build. Screenshot might not be that interesting.
I'm testing on CNN and the low-res tiles are much better
Attachment #8631306 - Flags: review?(blassey.bugs) → review-
Tested this on my N6 and if I'm understanding this correctly, I like the fading in more. 

Even if through measurement, loading the lower res tiles is slightly faster, I think there's something to be said about perceived performance here. It even feels smoother to me.
Flags: needinfo?(alam)
On single core devices (Nexus S Android 4.0) this just does not work. The tiles don't fade in just pop.
Assignee: nobody → snorp
Ignore the weird urlbar issue on the fading one, that seems to be some regression in Nightly when using synthetic input events.
We want to use an A/B test to determine if replacing the low-res tiles with fading ones will improve the experience.
Summary: Disable low-res tiles, enable fade-in of high-res tiles → Add an A/B test to for low-res tiles vs. fading tiles
Attachment #8631306 - Attachment is obsolete: true
Attachment #8631306 - Flags: review?(mark.finkle)
Attachment #8659942 - Flags: feedback?(mark.finkle)
Comment on attachment 8659942 [details] [diff] [review]
Add A/B experiment for low-res tiles vs. fading on Android

This looks like a good approach. I would use ExperimentHelper for Gecko-based pref experiments. Most Java/UI experiments will have the code in to UI code itself. Not to say we might not find a bigger use for ExperimentHelper.

I would rename runABExperiments() -> setup()

This is a good start.

If you add some NIGHTLY_BUILD support in there, I can add the segmentation to the switchboard server and we can make this live.
Attachment #8659942 - Flags: feedback?(mark.finkle) → feedback+
Another thing we should think about is the metrics for the experiment. What can we measure to see if we get a positive improvement. We are gearing up to get churn metrics, but those can be difficult to correlate back to a specific experiment. We'd like some simple data we can send to telemetry.
In the future, we could even do further this experiment by looking at more than just fading vs low res.

From convos with snorp on IRC, for example, we could test different ways to optimize perceived perforance through changes in fading duration, linear vs non-linear fading tweens, etc.
I've revived the code that makes fading tiles work, and I would like to turn it on by default in Nightly only. I am not sure what benefit we would really get from a switchboard experiment, and I would like to get the most testing possible with the fading tiles. Anthony does this sound ok to you?
Flags: needinfo?(alam)
(In reply to James Willcox (:snorp) (jwillcox@mozilla.com) from comment #16)
> I've revived the code that makes fading tiles work, and I would like to turn
> it on by default in Nightly only. I am not sure what benefit we would really
> get from a switchboard experiment, and I would like to get the most testing
> possible with the fading tiles. Anthony does this sound ok to you?

Sounds fine by me. Let's make sure we're letting everyone else know too (like when we turned on gecko carets for Nightly users) so we can get more feedback :D
Flags: needinfo?(alam) → needinfo?(bbermes)
https://hg.mozilla.org/mozilla-central/rev/82d27079b8f1
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 49
To be honest, as per Anthony's comment in 6, I'm having issues being persued that this is better performance. 

It's been more "annoying" and disturbing to me to get the black screen fading in and out and stopping me from "thinking" I'm continuing my browsing experience. Obviously being subjective here, and I'd like to get more users giving feedback.
Flags: needinfo?(bbermes)
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.