Closed Bug 732564 Opened 12 years ago Closed 12 years ago

[maple] Get viewport handling good again

Categories

(Firefox for Android Graveyard :: General, defect)

All
Android
defect
Not set
normal

Tracking

(blocking-fennec1.0 beta+)

RESOLVED FIXED
Firefox 14
Tracking Status
blocking-fennec1.0 --- beta+

People

(Reporter: kats, Assigned: kats)

References

Details

(Whiteboard: maple)

Attachments

(16 files, 34 obsolete files)

6.13 KB, patch
kats
: review+
Details | Diff | Splinter Review
1.29 KB, patch
kats
: review+
Details | Diff | Splinter Review
4.20 KB, patch
kats
: review+
Details | Diff | Splinter Review
7.46 KB, patch
kats
: review+
Details | Diff | Splinter Review
3.87 KB, patch
kats
: review+
Details | Diff | Splinter Review
4.90 KB, patch
kats
: review+
Details | Diff | Splinter Review
6.43 KB, patch
kats
: review+
Details | Diff | Splinter Review
3.51 KB, patch
kats
: review+
Details | Diff | Splinter Review
6.93 KB, patch
kats
: review+
Details | Diff | Splinter Review
5.70 KB, patch
kats
: review+
Details | Diff | Splinter Review
1.94 KB, patch
kats
: review+
Details | Diff | Splinter Review
21.36 KB, patch
kats
: review+
Details | Diff | Splinter Review
5.38 KB, patch
kats
: review+
Details | Diff | Splinter Review
6.15 KB, patch
kats
: review+
Details | Diff | Splinter Review
2.48 KB, patch
kats
: review+
Details | Diff | Splinter Review
5.13 KB, patch
kats
: review+
Details | Diff | Splinter Review
Make the viewport work as per The Plan(TM). The Plan(TM) can be found at https://wiki.mozilla.org/Fennec/NativeUI/Viewport
I'm leaving the paintingSuppressed hook on MetadataProvider in for now in case we need it for suppression during rotation, but if we end up not needing it at all then I can take it out.
Attached patch Part 2 - Remove JS-driven redraw (obsolete) — Splinter Review
This patch is a dupe of https://bug731897.bugzilla.mozilla.org/attachment.cgi?id=601976 but it needs to go after we remove paint suppression or things may break.
This implements the goal of bug 731897. That bug can be duped to here as well, I think.
This method of doing draw suppression during rotation is not very robust or clean. Taking it out and we can implement this behaviour better later if it's needed.
Attachment #602483 - Attachment is patch: true
So with these patches there's still a bunch of weird behaviour. One of the main ones is how gecko's paint suppression code works. That is, it doesn't really suppress painting; it just suppresses painting of new documents for 250ms (or some preffed value) during some point in the page-loading period (http://mxr.mozilla.org/mozilla-central/source/layout/base/nsPresShell.cpp#1914).

What this means is that:
(1) if gecko is asked to paint, it will paint the *old* document instead of the new one during this period. (see http://mxr.mozilla.org/mozilla-central/source/layout/generic/nsSubDocumentFrame.cpp#294). If we have already set the resolution on the pres shell (which we do currently) then this results in a visual glitch.
(2) if we haven't yet processed the meta viewport by that point, the page will glitch when we do process it and change the CSS viewport dimensions.

These appear to be the main reasons we have augmented paint suppression in browser.js.
Blocks: land-maple
Apologies in advance for flooding your review queue...
Attachment #602462 - Attachment is obsolete: true
Attachment #603841 - Flags: review?(chrislord.net)
Attached patch Part 2 - Remove JS-driven redraw (obsolete) — Splinter Review
Attachment #602463 - Attachment is obsolete: true
Attachment #603842 - Flags: review?(chrislord.net)
Attachment #602467 - Attachment is obsolete: true
Attachment #603845 - Flags: review?(chrislord.net)
Attachment #602469 - Attachment is obsolete: true
Attachment #602660 - Attachment is obsolete: true
Attachment #603849 - Flags: review?(chrislord.net)
Attachment #602662 - Attachment is obsolete: true
Attachment #603854 - Flags: review?(chrislord.net)
Attachment #602663 - Attachment is obsolete: true
Attachment #602664 - Attachment is obsolete: true
Attachment #603856 - Flags: review?(chrislord.net)
So a few notes:
- A lot of the changes aren't really stand-alone in that they will introduce regressions without future patches in the sequence. However I left them as separate patches for ease of review; I'll probably fold everything into a single patch on landing rather than try to figure out which subsets of patches can be folded together.

- I think additional patches can be created to further optimize this behaviour (for example I think some of the events are sending properties that are now not being used any more). However those will not impact correctness and can be done as separate bugs to prevent this one from ballooning even more out of control.

- I have a build with all these patches applied (plus a patch that only touches logging) posted at people.mozilla.org/~kgupta/tmp/viewports.apk - currently it's based on an older maple revision but I'm building now with the patches applied to 4012267a4f08 and will updated that apk in-place with the new build.

- The wiki page at https://wiki.mozilla.org/Fennec/NativeUI/Viewport describes much of the design and caveats that go along with these patches. If you want to see any changes to that page let me know (or go ahead and edit the page directly). Also, the code tries to use terms that I defined in that document.
Comment on attachment 603841 [details] [diff] [review]
Part 1 - Kill paint suppression code in browser.js

Review of attachment 603841 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine.
Attachment #603841 - Flags: review?(chrislord.net) → review+
Comment on attachment 603842 [details] [diff] [review]
Part 2 - Remove JS-driven redraw

Review of attachment 603842 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine.
Attachment #603842 - Flags: review?(chrislord.net) → review+
Comment on attachment 603844 [details] [diff] [review]
Part 3 - Merge update events from JS to Java since they do the same thing

Review of attachment 603844 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine.

::: mobile/android/base/gfx/GeckoLayerClient.java
@@ +286,5 @@
>      /** Implementation of GeckoEventResponder. */
>      public String getResponse() {
>          // We are responding to the events handled in handleMessage() above with
> +        // the display port margins we want. Note that all messages we are currently
> +        // handling (Viewport:Update) require this

Would prefer to just reformat the whole comment rather than have this shorter second line. Nit-picking.
Attachment #603844 - Flags: review?(chrislord.net) → review+
Comment on attachment 603845 [details] [diff] [review]
Part 4 - Some function rearranging

Review of attachment 603845 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with comment addressed.

::: mobile/android/chrome/content/browser.js
@@ -1651,5 @@
>      };
>  
> -    // Update the viewport to current dimensions
> -    viewport.x = this.browser.contentWindow.scrollX || 0;
> -    viewport.y = this.browser.contentWindow.scrollY || 0;

Would rather not remove this, and s/Update/Set/ in the comment. Either that, or add the transform in-line too, but that'd look pretty messy...
Attachment #603845 - Flags: review?(chrislord.net) → review+
Comment on attachment 603847 [details] [diff] [review]
Part 5 - Get rid of the updateViewport function

Review of attachment 603847 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine.

::: mobile/android/chrome/content/browser.js
@@ +2029,2 @@
>      // Ensure that when making changes to this function that code path
> +    // is not accidentally removed (the call to sendViewportUpdate() is at the

put at the on the next line, for neatness?
Attachment #603847 - Flags: review?(chrislord.net) → review+
Comment on attachment 603849 [details] [diff] [review]
Part 6 - Distinguish between updates that go to Java directly and that go via compositor

Review of attachment 603849 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine.
Attachment #603849 - Flags: review?(chrislord.net) → review+
Comment on attachment 603852 [details] [diff] [review]
Part 7 - Have the Viewport:Update message synchronously update the java viewport

Review of attachment 603852 [details] [diff] [review]:
-----------------------------------------------------------------

Not really sure how the page size stays in sync after this patch, but I assume that this is dealt with in a later patch.

::: mobile/android/base/gfx/GeckoLayerClient.java
@@ +255,5 @@
> +                synchronized (mLayerController) {
> +                    ViewportMetrics currentMetrics = new ViewportMetrics(mLayerController.getViewportMetrics());
> +                    currentMetrics.setOrigin(newMetrics.getOrigin());
> +                    currentMetrics.setZoomFactor(newMetrics.getZoomFactor());
> +                    currentMetrics.setPageSize(newMetrics.getPageSize());

Copying currentMetrics seems to be overkill just to retain the viewport size (and also isn't particularly clear without a comment).
Attachment #603852 - Flags: review?(chrislord.net) → review+
Comment on attachment 603854 [details] [diff] [review]
Part 8 - Take out java-side draw suppression during rotation

Review of attachment 603854 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine and I always thought this code was a little suspect...
Attachment #603854 - Flags: review?(chrislord.net) → review+
Comment on attachment 603856 [details] [diff] [review]
Part 9 - Ignore user actions and view update events if browser content doc is not displayed

Review of attachment 603856 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with comment addressed.

::: mobile/android/chrome/content/browser.js
@@ +379,5 @@
>      CharacterEncoding.uninit();
>    },
>  
> +  // This function returns true during periods where the browser displayed document is
> +  // different from the browser content document, so user actions and some kinds of viewport

If this comment is correct, the function needs to be renamed - I would expect the opposite behaviour (returning false), going on the function name.
Attachment #603856 - Flags: review?(chrislord.net) → review+
Comment on attachment 603857 [details] [diff] [review]
Part 10 - Move the document-shown event to be a before-first-paint event

Review of attachment 603857 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not the right person to review this, though it looks fine to me.
Attachment #603857 - Flags: review?(chrislord.net)
Attachment #603857 - Flags: review?(bzbarsky)
Attachment #603857 - Flags: review+
Comment on attachment 603860 [details] [diff] [review]
Part 11 - We need to clamp the viewport when animations are aborted to avoid overscroll

Review of attachment 603860 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me.
Attachment #603860 - Flags: review?(chrislord.net) → review+
Comment on attachment 603867 [details] [diff] [review]
Part 12 - Move the displayport stuff to Java and rework how it's done

Review of attachment 603867 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.

::: mobile/android/base/gfx/GeckoLayerClient.java
@@ +92,3 @@
>          mScreenSize = new IntSize(0, 0);
>          mBufferSize = new IntSize(0, 0);
> +        mDisplayPort = new RectF(0, 0, 0, 0);

though it's consistent with the other two, none of the initialisation parameters are needed here.
Attachment #603867 - Flags: review?(chrislord.net) → review+
Comment on attachment 603868 [details] [diff] [review]
Part 13 - Update the display port when we get a new first-paint viewport

Review of attachment 603868 [details] [diff] [review]:
-----------------------------------------------------------------

Looks ok. Although the comments are already good and lengthy, I think perhaps they could explain a bit more (but this is still r+ regardless). It took me a while to get my head around it, and I'm not sure if they're really understandable without a good knowledge of how the whole thing works.
Attachment #603868 - Flags: review?(chrislord.net) → review+
Comment on attachment 603870 [details] [diff] [review]
Part 14 - Clean out the view resize handling codepath

Review of attachment 603870 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.

::: mobile/android/base/gfx/GeckoLayerClient.java
@@ +210,5 @@
>      void viewportSizeChanged() {
> +        // here we send gecko a resize message. The code in browser.js is responsible for
> +        // picking up on that resize event, modifying the viewport as necessary, and informing
> +        // us of the new viewport.
> +        sendResizeEventIfNecessary(true);

Do we ever call sendResizeEventIfNecessary(false) anymore?
Attachment #603870 - Flags: review?(chrislord.net) → review+
Excellent work :)
I've updated my local patches as described below.

> Part 3 - Merge update events from JS to Java since they do the same thing
> 
> Would prefer to just reformat the whole comment rather than have this
> shorter second line. Nit-picking.

Fixed.

> Part 4 - Some function rearranging
> 
> Would rather not remove this, and s/Update/Set/ in the comment. Either that,
> or add the transform in-line too, but that'd look pretty messy...

Moved the x and y assignment back out of the object declaration, and updated the comment.

> Part 5 - Get rid of the updateViewport function
> 
> put at the on the next line, for neatness?

Fixed.

> Part 7 - Have the Viewport:Update message synchronously update the java
> viewport
> 
> Not really sure how the page size stays in sync after this patch, but I
> assume that this is dealt with in a later patch.

It's actually dealt with in bug 732091, which starts calling setPageSize on GeckoLayerClient from the compositor as it finds out about updates to the page size. This replaces the previous path which updated the page size in beginDrawing()/endDrawing().

> Copying currentMetrics seems to be overkill just to retain the viewport size
> (and also isn't particularly clear without a comment).

This was a result of me not properly taking into account the introduction of ImmutableViewportMetrics. I've rearranged this so that we just apply the size from the layer controller's metrics to the new JSON-created metrics, and then set that as the metrics on the layer controller. This avoids the unnecessary copying. I also added a comment to clarify.

> Part 9 - Ignore user actions and view update events if browser content doc
> is not displayed
> 
> If this comment is correct, the function needs to be renamed - I would
> expect the opposite behaviour (returning false), going on the function name.

Whoops, the comment was out of date. I changed it say return false and also clarified when the period starts and ends, rather than referring to a "flag".

> Part 12 - Move the displayport stuff to Java and rework how it's done
> 
> though it's consistent with the other two, none of the initialisation
> parameters are needed here.

I took out the parameters on the line I added, but left the others as-is.

> Part 13 - Update the display port when we get a new first-paint viewport
> 
> Looks ok. Although the comments are already good and lengthy, I think
> perhaps they could explain a bit more (but this is still r+ regardless). It
> took me a while to get my head around it, and I'm not sure if they're really
> understandable without a good knowledge of how the whole thing works.

I rewrote the comments so that hopefully they are more clear; will upload updated patch.

> Part 14 - Clean out the view resize handling codepath
> 
> Do we ever call sendResizeEventIfNecessary(false) anymore?

We do still call it during initialization, and I think it might still be required there because (if I understand the code right) there's a race between the GeckoLayerClient coming up and the FlexibleGLSurfaceView getting the surfaceChanged notification, and having that call ensures we always set the size properly.
Updated the comments in the patch; carrying r+
Attachment #603868 - Attachment is obsolete: true
Attachment #604061 - Flags: review+
Comment on attachment 603857 [details] [diff] [review]
Part 10 - Move the document-shown event to be a before-first-paint event

s/Move/Rename/ and corresponding other wording changes in the checkin comment?  

And fix the C++ comments that still refer to document-shown, please.

r=me with that.
Attachment #603857 - Flags: review?(bzbarsky) → review+
blocking-fennec1.0: --- → beta+
During the onLocationChange handler, the documentElement of the document is sometimes null. This resulted in the display port not getting set for the first paint, and the page would end up getting painted blurry because of a zoom level mismatch. This moves the call to setDisplayPort to just before the first paint so that this doesn't happen. (This should be folded into one of the previous patches, but keeping it separate for easier review).
Attachment #604205 - Flags: review?(chrislord.net)
Comment on attachment 604205 [details] [diff] [review]
Part 15 - Move first call to setDisplayPort

Review of attachment 604205 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me.
Attachment #604205 - Flags: review?(chrislord.net) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/d8946e029410 adds a dependency on the document-shown event, so unfortunately I can't just get rid of it. It was first added in January, so it's already in beta and will be out the door shortly, and taking it out seems like it might break things that will come to depend on it. So instead I'm leaving it as-is and just adding the before-first-paint event on top of it. Re-requesting review.
Attachment #603857 - Attachment is obsolete: true
Attachment #604431 - Flags: review?(bzbarsky)
This fixes us rendering at odd zooms sometimes on rotation of a meta-viewport page (e.g. runfield). Also if you load a meta-viewport page after a non-meta-viewport page or vice-versa, we now reset the zoom and recalculate it properly.
Attachment #604456 - Flags: review?(chrislord.net)
Comment on attachment 604456 [details] [diff] [review]
Part 16 - Ensure zoom from meta-viewport is correctly calculated

Review of attachment 604456 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me, I think...
Attachment #604456 - Flags: review?(chrislord.net) → review+
So I'm just doing a review for bug 732091 and I have a question that seems like it should go here.

From what I can tell currently the first paint code is only used when switching between tabs. But it seems like the intention is to also use it while loading new pages. As https://wiki.mozilla.org/Fennec/NativeUI/Viewport points out (under Paint suppression) this is a  bit tricky. That page says "we detect the point at which the new document is about to be painted, and perform any necessary actions". I haven't been able to find this code. Did I overlook it? Or is it still upcoming?
Parts 6 and 10 contain the relevant code. In part 6, the document-shown event handler sets the first paint flag by calling displayedDocumentChanged(). In part 10 this code is changed to run on receipt of the before-first-paint event, which gets fired before the first paint of a newly loading page instead of the document-shown event which gets triggered after the paint.
Ah, ok, thank you.
Comment on attachment 604431 [details] [diff] [review]
Part 10 - Add a before-first-paint event

>+  nsBeforeFirstPaintDispatcher(nsCOMPtr<nsIDocument> aDocument)

Argument here should be just nsIDocument*.

r=me with that fixed, I guess, but check with the telemetry folks whether they really need the old event.  I'd really prefer to not fire them both...
Attachment #604431 - Flags: review?(bzbarsky) → review+
> r=me with that fixed, I guess, but check with the telemetry folks whether
> they really need the old event.  I'd really prefer to not fire them both...


d8946e029410 was backed out and I did not see any other uses in MXR. Chances are also good that no add-ons are using "document-shown" yet.
Add r=Cwiiis to patch; no additional changes. Carrying r+
Attachment #603841 - Attachment is obsolete: true
Attachment #604933 - Flags: review+
Attachment #603842 - Attachment is obsolete: true
Attachment #604934 - Flags: review+
Minor update to formatting of comments as per review comments. Add r=Cwiiis to patch. Carrying r+
Attachment #603844 - Attachment is obsolete: true
Attachment #604935 - Flags: review+
Rebase on latest maple. A recent m-c merge introduced a bunch of code that used tab.viewport which all needed to be changed to tab.getViewport(). Also updated with review comments and added r=Cwiiis to patch description.
Attachment #603845 - Attachment is obsolete: true
Attachment #604938 - Flags: review+
Fix comment formatting as per review comments.
Attachment #603847 - Attachment is obsolete: true
Attachment #604939 - Flags: review+
Rebase to account for how isFirstPaint is a boolean attribute on DOMWindowUtils rather than a function now.
Attachment #603849 - Attachment is obsolete: true
Attachment #604940 - Flags: review+
Update with review comments (remove extra ViewportMetrics creation).
Attachment #603852 - Attachment is obsolete: true
Attachment #604941 - Flags: review+
Updated some code comments as per review comments.
Attachment #603856 - Attachment is obsolete: true
Attachment #604947 - Flags: review+
Change the nsCOMPtr<nsIDocument> to just an nsIDocument* as per bz's review comment. Also I talked to the Neil and he said he needs document-shown rather than before-first-paint so leaving that as-is.
Attachment #604431 - Attachment is obsolete: true
Attachment #604950 - Flags: review+
(In reply to Kartikaya Gupta (:kats) from comment #73)
> Also I talked to the Neil and he said he needs document-shown

Err, that should be "to Neil" (from the telemetry team).
Minor updates for review comments.
Attachment #603867 - Attachment is obsolete: true
Attachment #604953 - Flags: review+
Somehow got the wrong bug number on the last patch. Backed out and relanded with the right bug number.

https://hg.mozilla.org/projects/maple/rev/d8326d63c320
https://hg.mozilla.org/projects/maple/rev/e3798e906917
Keywords: qawanted
Depends on: 735029
Keywords: qawanted
Target Milestone: --- → Firefox 14
Depends on: 752799
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: