Closed Bug 1487542 Opened Last year Closed 11 months ago

DuckDuckGo sends flag 0 when clicking on search results

Categories

(GeckoView :: General, defect, P2)

defect

Tracking

(geckoview62 wontfix, geckoview64 wontfix, firefox-esr52 wontfix, firefox-esr60 wontfix, firefox61 wontfix, firefox62 wontfix, firefox63 wontfix, firefox64 wontfix, firefox65 fixed)

RESOLVED FIXED
mozilla65
Tracking Status
geckoview62 --- wontfix
geckoview64 --- wontfix
firefox-esr52 --- wontfix
firefox-esr60 --- wontfix
firefox61 --- wontfix
firefox62 --- wontfix
firefox63 --- wontfix
firefox64 --- wontfix
firefox65 --- fixed

People

(Reporter: ekager, Assigned: mbrubeck)

References

(Depends on 2 open bugs)

Details

(Whiteboard: [geckoview:klar:p2])

Attachments

(1 file, 2 obsolete files)

In Focus we use GeckoSession.NavigationDelegate.LOAD_REQUEST_IS_USER_TRIGGERED to know when the user has navigated away from search results (to show the search terms instead of a search URL in the URL Bar). When searching with DDG the flags sent to onLoadRequest are 0 when clicking on a search result link.
What is the expected flag value? What does the user experience when clicking the DDG search results?
Expected flag value is LOAD_REQUEST_IS_USER_TRIGGERED. It should indicate the user navigated away from a page. User will then be able to see the URL in the URL bar for the page they clicked on. 
 https://mozilla.github.io/geckoview/javadoc/mozilla-central/org/mozilla/geckoview/GeckoSession.NavigationDelegate.html#LOAD_REQUEST_IS_USER_TRIGGERED
Flags: needinfo?(ekager)
Assignee: nobody → mbrubeck
Priority: -- → P1
Whiteboard: [geckoview:klar:p1]
`NavigationDelegate.LOAD_REQUEST_IS_USER_TRIGGERED` is set based on `nsIDocShell::INTERNAL_LOAD_FLAGS_IS_USER_TRIGGERED`, which is set only in `nsDocShell::OnLinkClick`.  This means it is only set when navigating to the `href` URI of an HTML link.  It is not set when navigation is triggered by script, even in cases like this where the script overrides a link click event:

    <a href="#">test</a>
    <script>
      document.querySelector("a").addEventListener("click", function(e) {
        e.preventDefault();
        window.location = "https://example.com/";
      });
    </script>

We either need to track down other callers of `LoadURI` and have them set the flag too, or move the `EventStateManager::IsHandlingUserInput` check into GeckoView code.

I think the relevant caller for this case is:

https://searchfox.org/mozilla-central/source/dom/base/Location.cpp#232
Status: NEW → ASSIGNED
Depends on: 1439013
Attached patch WIP patch (obsolete) — Splinter Review
This fixes the bug for the simple test case above, but it doesn't work for DuckDuckGo.  It appears that DuckDuckGo's navigation runs in a JS timeout, and `IsHandlingUserInput` is no longer true by the time it calls `LoadURI`.
This is related to bug 1185052.
Blocks: 1439013
Depends on: 1185052
No longer depends on: 1439013
Based on bug 1185052, it seems that we can't reliably use `IsHandlingUserInput` to determine whether script is running as a result of a user action.  It doesn't work at all if the script involves asynchronous callbacks (e.g. timeouts or network requests).

The use case for this flag is to determine whether a user has navigated away from a search results page. Currently Focus considers user-initiated link clicks to be navigating away, but not page loads that are triggered by other things like redirects.

Can we come up with a more reliable method or heuristic to determine which page loads to treat as "navigating away" from the search results?  For example, can we detect redirecs more explicitly?  Or look at the domain being loaded?
Some possible solutions:

* Make EventStateManager track user input state across timeouts (i.e., fix bug 1185052).  This is highly desired, but it requires "non-trivial" work according to Xidorn's comments in that bug.  Might not be feasible in the Focus + GV time frame.

* Use other attributes of the request to determine whether it should be used or ignored for the purpose of clearing search terms in the Focus location bar.  To do this, we'd need examples of the cases where a request should be ignored.

* Help DuckDuckGo change their JavaScript to work around this bug.  This would require us to reach out to DuckDuckGo, and for them to develop and deploy a solution in time.

* Always clear search terms when loading a new URI.  This might regress the "search terms" feature in some cases, causing the URI to be displayed instead.

* Never show search terms in the address bar.  This would regress the "search terms" feature in all cases.
Notes from GeckoView triage:

We think the best fix for this in GeckoView is to add a flag to load events caused by redirects, and for Focus to use this flag as an alternate way to determine which page loads should clear search terms.  We'll implement this in nightly.

However, this fix may be too involved to land in GV 62 and release with Focus 7.0.  For that release, we'd need a short-term mitigation in Focus.  Suggested workaround: *If* the search engine is DuckDuckGo, ignore the USER_TRIGGERED flag and instead clear search terms on *all* page loads.

Emily, can the Focus team implement this temporary DDG-specific workaround? If so, can you file a Focus bug and link it here, please?
Flags: needinfo?(ekager)
Product input (Vesta/Barbara): disabling the search term feature is fine; for DDG or across the board.
Here's the issue to disable the feature for DDG + GV 
https://github.com/mozilla-mobile/focus-android/issues/3401
Flags: needinfo?(ekager)
status-geckoview62=fix-optional because Vesta says this bug is not a Focus+GV blocker. However, the Focus team does not want to disable the search term feature for DDG for Focus 7.0, but maybe in Focus 7.1 if we don't have a GV fix in the 1-2 weeks.
Whiteboard: [geckoview:klar:p1] → [geckoview:klar:p2]
Emily has a fix to temporarily disable DDG search terms until this GV bug is fixed:

https://github.com/mozilla-mobile/focus-android/issues/3401
status-geckoview62=wontfix because Matt says Focus 7.0 thanks to Emily's workaround (comment 12).

Once this GV bug is fixed, Emily can remove her workaround in this Focus issue:

https://github.com/mozilla-mobile/focus-android/issues/3256
This depends on bug 1478171, which got backed out along with bug 1441059.
Depends on: 1478171, 1441059
Attached patch patch v2 (obsolete) — Splinter Review
Instead of trying to fix the USER_TRIGGERED flag, this adds an IS_REDIRECT flag.  This depends on the temporarily backed-out patch from bug 1478171.

After this new flag is added, instead of clearing search terms when USER_TRIGGERED is set, Focus should clear search terms when IS_REDIRECT is *not* set.

I could still use some test cases to verify this filters out the appropriate events.  Emily, do you know which search engines had redirects (or other non-user-triggered loads) that should *not* cause Focus to clear search terms?
Attachment #9007023 - Attachment is obsolete: true
Flags: needinfo?(ekager)
Sorry for the late reply! IIRC one example is using Wikipedia as your search engine - they redirect to the article page directly if the search term matches the article title (e.g. search for "Fennec").
Flags: needinfo?(ekager)
Attachment #9015340 - Attachment is obsolete: true
Pushed by mbrubeck@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/31b5c6b8ef22
Change LoadRequest.isUserTriggered to isRedirect. r=snorp,smaug
Flags: needinfo?(mbrubeck)
Pushed by mbrubeck@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6893246437a2
Change LoadRequest.isUserTriggered to isRedirect. r=snorp,smaug
https://hg.mozilla.org/mozilla-central/rev/6893246437a2
Status: ASSIGNED → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 65
status-geckoview64=wontfix
Product: Firefox for Android → GeckoView
Target Milestone: Firefox 65 → mozilla65
You need to log in before you can comment on or make changes to this bug.