1496684 - Fennec GeckoView hangs if Gecko isn't already running (with extension using blocking webRequest listener)

Status: VERIFIED FIXED

(Firefox for Android Graveyard :: Custom Tabs, defect, P2)

Description

[Paul [pwd]]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0

Steps to reproduce:

These have stopped working over the past couple of weeks

Confirmations: https://www.reddit.com/r/firefox/comments/9ldetu/firefox_custom_tabs_not_loading/

Comment 1

[Sorina Florean [:sflorean]]

Thanks for the report!
Bug 1484977 is already filed for this issue (custom tabs). Please comment/follow the discussions there. 
As for page shortcuts, bug 1480793 was fixed recently. Thanks!

Comment 2

[Jan Henning [:JanH]]

Bug 1484977 wasn't a regression, whereas this sounds like one, in which case it wouldn't be the same issue.
I think I could reproduce it a few days ago, but at that time http://archive.mozilla.org/ had problems and so mozregression wasn't working. After updating yesterday, I can no longer reproduce this again, though.

Comment 3

[Paul [pwd]]

Indeed, custom tabs were working perfectly and then randomly stopped, so not a dupe of bug 148977. 

I reinstalled and some custom tabs are working again. I need to continue testing for a difinitive answer there. 

That said, page shortcuts definitely aren't working. I had assumed that the problem was related to the custom tab problem, but no matter how many times I create the shortcut for https://my.hivehome.com/dashboard it doesn't appear.

Comment 4

[Paul [pwd]]

So my further testing has uncovered that unless Firefox has been started, the custom tabs don't initiate.

Comment 5

[Jan Henning [:JanH]]

[Tracking Requested - why for this release]: Custom tabs are broken when Firefox isn't already running in background

(In reply to Paul [pwd] from comment #4)
> So my further testing has uncovered that unless Firefox has been started,
> the custom tabs don't initiate.

Good point, that must have been why I couldn't reproduce yesterday. Interestingly enough, about: pages seem unaffected - these always open successfully in a custom tab even when Gecko has start from scratch.

As for page shortcuts - your example page is working fine for me, both through the PWA and the normal page shortcut route. If you can still reproduce, please open a new bug and include your phone model and OS version.

Comment 6

[Julien Cristau [:jcristau]]

That sounds pretty bad.  Do we know when this stopped working?

Comment 7

[Sorina Florean [:sflorean]]

(In reply to Paul [pwd] from comment #3)
> 
> That said, page shortcuts definitely aren't working. I had assumed that the
> problem was related to the custom tab problem, but no matter how many times
> I create the shortcut for https://my.hivehome.com/dashboard it doesn't
> appear.

Tried to reproduce this on the latest nightly build with Motorola Nexus 6(Android 7.1.1), Google Pixel(Android 9), LG G4(Android 6.0.1) and Xiaomi Mi4i(Android 5.0.2) but with no success. Do you have some additional steps in order to re-test and find a regression window? Thanks!

Comment 8

[Paul [pwd]]

(In reply to Sorina Florean [:sorina] from comment #7)
> (In reply to Paul [pwd] from comment #3)
> > 
> > That said, page shortcuts definitely aren't working. I had assumed that the
> > problem was related to the custom tab problem, but no matter how many times
> > I create the shortcut for https://my.hivehome.com/dashboard it doesn't
> > appear.
> 
> Tried to reproduce this on the latest nightly build with Motorola Nexus
> 6(Android 7.1.1), Google Pixel(Android 9), LG G4(Android 6.0.1) and Xiaomi
> Mi4i(Android 5.0.2) but with no success. Do you have some additional steps
> in order to re-test and find a regression window? Thanks!

Strangely, I managed to reproduce this twice earlier.

First time, I was in Flamingo, I saw an article, I pressed it to read some more info and nothing happened (I was taken to the custom tab and the page remained blank), so I went into nightly manually, waited for whatever my last tab was to load, went back into Flamingo and voila!

Second time, replace Flamingo with Tumblr.

LG G6, Android 8.0.0 (August Security Patches)

Comment 9

[Marcia Knous [:marcia]]

Susheel - This seems like a bad regression and is blocking 64 - can someone please take a look?

Comment 10

[:sdaswani]

Vlad can you look at this ASAP.

Comment 11

[Vlad Baicu]

I also haven't had much luck reproducing this, considering that next week I am going to be on PTO, one of my colleagues is going to pick this one up Monday as a top priority.

Comment 12

[Paul [pwd]]

(In reply to Vlad Baicu [on PTO until 10.29] from comment #11)
> I also haven't had much luck reproducing this, considering that next week I
> am going to be on PTO, one of my colleagues is going to pick this one up
> Monday as a top priority.

I have the tab queue enabled if that helps.

Another STR:
1. Go to Google+
2. Find a post with a link
3. Tap link to open in custom tab
4. Wait forever!

Comment 13

[Petru-Mugurel Lingurar [:petru]]

This seems to be a problem with Gecko.. View?
The issue seems similar to bug 1494748, bug 1497963 and also https://github.com/mozilla-mobile/focus-android/issues/3679 from Focus

STR to reproduce this 100% on latest Nightly:
1 - Have Fennec opened, have some pages loaded
2 - Open link as custom tab in Fennec
3 - Remove Fennec from Recents
4 - Start Fennec 
5 - Observe
- previous loaded tabs now show blank images and cannot be convinced to reload / repaint
- new tabs seem to work
- doing a drag down gesture on a blank page shows underneath the content of the last tab, even though the url and title of that blank page is different than for the last page

Small video for this - https://drive.google.com/file/d/1evRF_St5WeVKzTT7U_gOUGlB0hhvRY0o/view?usp=sharing

NIing James as this seems more like a Gecko issue.

Comment 15

[James Willcox (:snorp) ([e-mail]) (he/him)]

It does look like we broke something here. Dylan, you were looking into similar problems, can you take this one?

Comment 16

[Dylan Roeh (:droeh) (he/him)]

Yeah, I was actually looking into this a bit on Monday afternoon. I thought it was going to just be an instance of failing to call coverUntilFirstPaint(Color.TRANSPARENT) as I think we've seen in some other cases, but it doesn't look like it's that simple: following the STR in comment 13, when I reach step 4 and restart Fennec, we do actually get the coverUntilFirstPaint(Color.TRANSPARENT) call as expected but still have a white screen. I'll investigate further.

Comment 17

[Petru-Mugurel Lingurar [:petru]]

:dietrich said he is able to reproduce a similar issue (with different STR) consistently and can help with debugging
https://bugzilla.mozilla.org/show_bug.cgi?id=1494748#c15

Comment 18

[Dylan Roeh (:droeh) (he/him)]

Clearing the regression and regressionwindow-wanted keywords; I've checked periodically back to October of last year and as far as I can tell, this *never* worked. If someone has an actual working build date, please let me know.

Comment 19

[Paul [pwd]]

Comment 13 confuses the hell out of me and for me personally, convolutes this bug, but using the STR in Comment 12, this was certainly working until recently.

Comment 20

[Dylan Roeh (:droeh) (he/him)]

(In reply to Paul [pwd] from comment #19)
> Comment 13 confuses the hell out of me and for me personally, convolutes
> this bug, but using the STR in Comment 12, this was certainly working until
> recently.

Interesting, there might be multiple issues here. I was basing my tests on comment 13. I'll be sure to take your STR into account as well.

Comment 21

[:sdaswani]

Adding NI for Dylan so we don't lose momentum on this.

Comment 22

[Dylan Roeh (:droeh) (he/him)]

(In reply to Dylan Roeh (:droeh) from comment #20)
> (In reply to Paul [pwd] from comment #19)
> > Comment 13 confuses the hell out of me and for me personally, convolutes
> > this bug, but using the STR in Comment 12, this was certainly working until
> > recently.
> 
> Interesting, there might be multiple issues here. I was basing my tests on
> comment 13. I'll be sure to take your STR into account as well.

Quick follow-up: I can't reproduce the issue seen in comment 12. Paul, if you have the time/inclination, logcat output from a custom tab that hangs on a white screen the way you're describing might be helpful.

Comment 23

[Paul [pwd]]

I've been busy today, but will try and grab a logcat for you tomorrow.

Comment 24

[Paul [pwd]]

To prompt myself.

Comment 25

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

David, this might be another issue for the geckoview team once we have more info from the reporter.

Comment 26

[Dylan Roeh (:droeh) (he/him)]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #25)
> David, this might be another issue for the geckoview team once we have more
> info from the reporter.

I'm currently looking into this.

Comment 27

[David Bolter [:davidb] (NeedInfo me for attention)]

Thanks Dylan. Removing my NI.

Comment 28

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

There is a patch up in bug 1494748 so if that lands we can verify in this bug as well.

Comment 29

[Paul [pwd]]

Attachment: 201811141146-logcat.txt

Sorry for the delay, I broke my system. Anyway, here's a logcat

Comment 30

[Julien Cristau [:jcristau]]

see bug 1494748 comment 54.

Comment 31

[Leo_sk]

An easy and consistent way to reproduce the bug:
1. Set nightly as default
2. In settings>privacy, enable clear data on exit and choose opened tabs
3. This enables the quit button in the firefox menu
4. Exit firefox using the quit button
5. Open a custom tab through some app

Observed behaviour: nothing loads in custom tab

Intended behaviour: Site should load

This problem has been present in nightly at least since October

Comment 32

[Paul [pwd]]

(In reply to Leo_sk from comment #31)
> An easy and consistent way to reproduce the bug:
> 1. Set nightly as default
> 2. In settings>privacy, enable clear data on exit and choose opened tabs
> 3. This enables the quit button in the firefox menu
> 4. Exit firefox using the quit button
> 5. Open a custom tab through some app
> 
> Observed behaviour: nothing loads in custom tab
> 
> Intended behaviour: Site should load
> 
> This problem has been present in nightly at least since October

I won't dispute whether or not that can reproduce the problem as there's enough eyes on this bug already. But I would like to try and avoid the perception that this is a user issue. i.e. users are clearing data on exit or swiping the app away. This is an app issue whereby if you leave Fennec for a while and expect to load content from the web, Fennec puts a complete halt to that. It's a horrible user experience.

Comment 33

[Leo_sk]

(In reply to Paul [pwd] from comment #32)
> (In reply to Leo_sk from comment #31)
> > An easy and consistent way to reproduce the bug:
> > 1. Set nightly as default
> > 2. In settings>privacy, enable clear data on exit and choose opened tabs
> > 3. This enables the quit button in the firefox menu
> > 4. Exit firefox using the quit button
> > 5. Open a custom tab through some app
> > 
> > Observed behaviour: nothing loads in custom tab
> > 
> > Intended behaviour: Site should load
> > 
> > This problem has been present in nightly at least since October
> 
> I won't dispute whether or not that can reproduce the problem as there's
> enough eyes on this bug already. But I would like to try and avoid the
> perception that this is a user issue. i.e. users are clearing data on exit
> or swiping the app away. This is an app issue whereby if you leave Fennec
> for a while and expect to load content from the web, Fennec puts a complete
> halt to that. It's a horrible user experience.

Well this was how ended up discovering it and posting on reddit, which i am grateful to you for reporting. The problem is that if one simply exits by back button, custom tabs will still work. Now depending on android version and variant and user settings, nightly can be cleared from background after screen locking or it may not, making it pretty inconsistent. If there is sure shot reproducible method, it becomes easier to test

Comment 34

[Chris Peterson [:cpeterson]]

(In reply to Dylan Roeh (:droeh) from comment #26)
> (In reply to Liz Henry (:lizzard) (needinfo? me) from comment #25)
> > David, this might be another issue for the geckoview team once we have more
> > info from the reporter.
> 
> I'm currently looking into this.

Dylan, do you have any updates on this Custom Tabs issue or "content area frozen" bug 1494748?

Comment 35

[Dylan Roeh (:droeh) (he/him)]

(In reply to Chris Peterson [:cpeterson] from comment #34)
> (In reply to Dylan Roeh (:droeh) from comment #26)
> > (In reply to Liz Henry (:lizzard) (needinfo? me) from comment #25)
> > > David, this might be another issue for the geckoview team once we have more
> > > info from the reporter.
> > 
> > I'm currently looking into this.
> 
> Dylan, do you have any updates on this Custom Tabs issue or "content area
> frozen" bug 1494748?

The patch currently up for bug 1494748 should resolve this as well, I'm just waiting for review.

Comment 36

[Leo_sk]

(In reply to Dylan Roeh (:droeh) from comment #35)

> The patch currently up for bug 1494748 should resolve this as well, I'm just
> waiting for review.

I currently have the latest nightly from playstore and the problem is still present. Will another update implement the patch?

Comment 37

[Dylan Roeh (:droeh) (he/him)]

(In reply to Leo_sk from comment #36)
> (In reply to Dylan Roeh (:droeh) from comment #35)
> 
> > The patch currently up for bug 1494748 should resolve this as well, I'm just
> > waiting for review.
> 
> I currently have the latest nightly from playstore and the problem is still
> present. Will another update implement the patch?

The patch hasn't landed yet, so unfortunately the problem is still present on nightly. I'd recommend watching bug 1494748 for the most up-to-date information.

Comment 39

[timur.yildirim55]

Hello, I opened the #1516922 bug and it seems that it's a duplicate of this bug here so i post my lastest experience here.


Since the last version (or maybe before didn't tested actually) the bug only appear if there's a Adblocker module enabled.

I mean i use uBlock Origin, disabled it - force closed lastest Nightly - go to "Sync for reddit" and open a link (so it open a custom tabs) - BOUM it's working ! 

But if i enable uBlock Origin - Force close lastest nightly - go to "Sync for reddit" and open a link - it's not working, it's staying blank...

Comment 40

[Chris Peterson [:cpeterson]]

The fix for bug 1494748 was just uplifted to Fennec 65 Beta, so that might improve things in Friday's 65.0b8 release.

Comment 42

[Paul [pwd]]

Can I just confirm that despite the patch from bug 149478 landing, this bug is still a major issue.

Comment 43

[Paul [pwd]]

Opps, bug 1494748 not bug 149478.

Comment 44

[Chris Peterson [:cpeterson]]

Dylan says this bug doesn't affect GV and is Fennec-specific. Dylan recommends we wait to see if the issue quiets down in Fennec 65 Beta.

Comment 45

[Leo_sk]

Is it implemented in nightly channell? If it has, then i can confirm its not fixed

Comment 46

[Chris Lord [:cwiiis]]

(In reply to Chris Peterson [:cpeterson] from comment #44)
> Dylan says this bug doesn't affect GV and is Fennec-specific. Dylan
> recommends we wait to see if the issue quiets down in Fennec 65 Beta.

Is this expected to be fixed? Because it definitely isn't running Nightly.

Comment 47

[timur.yildirim55]

Yep still not fixed with 65b08 neither with the latest  nightly (still saying that the but seems to only appears when a adblocker (unlock or what ever) is active)

Comment 48

[Leo_sk]

(In reply to timur.yildirim55 from comment #47)
> Yep still not fixed with 65b08 neither with the latest  nightly (still
> saying that the but seems to only appears when a adblocker (unlock or what
> ever) is active)

I disabled all the add ons and repeated the steps i mentioned in comment 31, and bug is still present

Comment 49

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Dylan are you still working on this or should we hand it off to the Fennec team?

Comment 50

[Dylan Roeh (:droeh) (he/him)]

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #49)

Dylan are you still working on this or should we hand it off to the Fennec team?

As far as I can see, any complete fix for this is going to be Fennec-specific and won't likely contribute to GV, so I'm going to unassign myself.

Comment 51

[Petru-Mugurel Lingurar [:petru]]

I think Jan's patch from bug 1494748 should also resolve this so this should be checked again after that patch gets merged.

Comment 52

[timur.yildirim55]

It's still not solved even if the bug 1494748 was fixed on the latest nightly .. when an Adblock module is active I can't have custom tabs working if I killed Firefox nightly from the recent app

Comment 53

[Jan Henning [:JanH]]

To be honest, personally I wouldn't have expected any improvement there from my patch, either.

Comment 54

[nandhp]

I can confirm that the problem goes away if I disable uBlock Origin. I was testing opening a link from the Gmail app in Firefox 64.0.2 (installed from Google Play Store) on Android 9 (Pie) on a Nokia 6.1. (I also have the same problem opening links from Reddit Sync, but didn't try that with the add-on disabled.)

Comment 56

[Jan Henning [:JanH]]

Bug 1520992 has some more info that would fit the theory about this happening in combination with certain extensions.

While testing something unrelated, yesterday I've also experienced the case where the initial load in a custom tab would in fact work, but subsequently clicking on a link would hang. So it seems there's another bug in that the very first request might not actually be handled by the extension, which is good for this bug, but another issue in itself.

Comment 57

[timur.yildirim55]

(In reply to nandhp from comment #54)

I can confirm that the problem goes away if I disable uBlock Origin. I was testing opening a link from the Gmail app in Firefox 64.0.2 (installed from Google Play Store) on Android 9 (Pie) on a Nokia 6.1. (I also have the same problem opening links from Reddit Sync, but didn't try that with the add-on disabled.)

With my oneplus 6t on android pie the bug of a non-loading custom tabs (tabs stay white trying to load infinitly) only appear if there's a adblocking module active and firefox nightly killed from recent.. If the adblocking module is disabled , custom tabs (from sync for reddit/ gmail / all other app that use custom tabs) works great even if Firefox nightly is killed from recent .. so yeah the issue is with module handling i think

Comment 58

[[e-mail]]

with extension using webRequest.onBeforeSendHeaders in blocking mode?

uBlock Origin does not install any onBeforeSendHeaders listener; it installs listeners only for onBeforeRequest and onHeadersReceived.

Comment 59

[Jan Henning [:JanH]]

I guess any blocking webRequest listener will do...

If somebody could bundle up a minimal testcase extension along the lines of bug 1520992 comment 0, this would be appreciated.

Comment 61

[timur.yildirim55]

@lizzard
As written Firefox nightly is affected too .. so 67 if affected too

Comment 64

[Jan Henning [:JanH]]

Attachment: [e-mail]

(In reply to Jan Henning [:JanH] from comment #59)

If somebody could bundle up a minimal testcase extension along the lines of bug 1520992 comment 0, this would be appreciated.

Never mind, I found something in the webextensions sample repository.

Comment 65

[Nameless Voice]

There seemed to be some confusion as to how to reproduce this bug earlier in its history, unsure if that's been clarified or not yet?

Anywhere, here are a very simple set of steps that I can use to reproduce it every time (assuming an extension such as uBlock Origin is active):

1. Press the "app switcher" virtual button on the phone (left button on Samsungs, right button on most other phones)
2. Swipe both Firefox and its custom apps away to quit them (if any are running)
3. Open the custom tab again

Comment 66

[Jan Henning [:JanH]]

Attachment: Bug 1496684 - Dispatch commonly expected startup notifications when opening a GeckoView window. r?snorp

Once a webextension using a blocking WebRequest listener has started loading,
all network connections covered by the extension's manifest are held until the
extension is ready the process them.
One condition for the extension being ready apparently includes browser startup
having progressed far enough, as signified by "browser-delayed-startup-finished"
having been dispatched.
Therefore, we have to start sending that notification when opening a new Gecko-
View window, too, and copy Fennec's InitLater() system for that.

Unlike Fennec, we cannot tie registration of those InitLater() runnables to the
initial content load having progressed far enough because of
a) e10s, which makes that approach neither easily possible nor really sensible,
as content will load in a different process in that case, and
b) because we're racing with extension startup here - if extensions are loaded
quick enough to block even the initial page load, we'd be deadlocked: We
cannot send the notification until the page finishes loading, but the page
cannot load until we send the notification. Fennec isn't affected by the
latter problem because "sessionstore-windows-restored", which Fennec will
send in any case, serves as an alternative pathway for completing extension
startup.

And unlike Desktop, we don't really have any chrome content to paint, so we
cannot tie delayed initialisation to a paint listener for that, either.

Therefore, we simply fire off a runnable at the end of geckoview.js's
startup() method, which should give more pressing initialisation tasks enough of
a headstart.

For completeness, we're also adding the "browser-idle-startup-tasks-finished"
notification and thereby solve bug 1465832 as well, allowing the ScriptPreloader
to detect which scripts are commonly loaded during GeckoView startup and to
start caching and pre-parsing them.

Comment 68

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/17c5a7677d9f
Dispatch commonly expected startup notifications when opening a GeckoView window. r=snorp

Comment 69

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/17c5a7677d9f

Comment 70

[timur.yildirim55]

So it's fixed on today nightly build ?

Comment 71

[Jan Henning [:JanH]]

Yes, should be.

Comment 72

[Martin Giger [:freaktechnik]]

It's working for me now in the latest Nightly available from the play store.

Comment 73

[timur.yildirim55]

Yeah seems solved for me too :) I'll make more test after work

Comment 74

[Jan Henning [:JanH]]

Comment on attachment 9041877 [details]
Bug 1496684 - Dispatch commonly expected startup notifications when opening a GeckoView window. r?snorp

Beta/Release Uplift Approval Request

Feature/Bug causing the regression

unclear

User impact if declined

For users with extensions using blocking WebRequest listeners, e.g. adblockers, pages in custom tabs/PWAs might not load if Firefox isn't already running.

Is this code covered by automated tests?

No

Has the fix been verified in Nightly?

Yes

Needs manual test from QE?

No

If yes, steps to reproduce

Install a corresponding extension, e.g. uBlock Origin, and then follow comment #65.

List of other uplifts needed

none

Risk to taking this patch

Low

Why is the change risky/not risky? (and alternatives if risky)

GeckoView was merely missing a notification that was expected by other platform code (including extension startup) to detect browser window initialisation.

String changes made/needed

none

Comment 75

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Verified in comments 72 and 73.

Comment 76

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 9041877 [details]
Bug 1496684 - Dispatch commonly expected startup notifications when opening a GeckoView window. r?snorp

Fix for a hang, verified in Nightly, let's uplift for beta 7.

Comment 78

[[Back July 11] Ryan VanderMeulen]

Comment on attachment 9041877 [details]
Bug 1496684 - Dispatch commonly expected startup notifications when opening a GeckoView window. r?snorp

Glad to see this going into Fx66, but I don't think this needs to be in the dot release.

Comment 79

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/releases/mozilla-beta/rev/95b884dcbd6f

Comment 80

[Stefan Deiac]

Verified on the latest version of Beta 66.0b7 using steps from comment 65.
Devices:
-OnePlus 5T (Android 9);
-Samsung Galaxy S8 (Android 8.0).

Due to that, I'll mark this issue as Verified, thanks.