1017078 - Record search activity for later use

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Data Providers, defect)

Description

[Richard Newman [:rnewman]]

We know when you type a search in the URL bar. Let's record when, what, which engine, etc. in a manner that's a little more accessible than scraping the history DB.

This will support search history in all kinds of ways.

Comment 1

[:Margaret Leibovic]

We can use this bug to explore creating a content provider for searches. It would be nice to store any search you make in Fennec, such that the search activity can pull from a richer set of history for suggestions.

We can also experiment with a follow-up to bootstrap this with searches that are stored in your history db, which we can figure out from search query templates.

However, we should probably get bug 1021864 underway before we get too deep into this.

Comment 2

[Eric Edens [:eedens]]

Hey Richard, regarding the "which engine" field, I think there are some useful things we could with this in the future, but we really haven't specced anything that would use it now. (At least with the search activity.)

Part of my concern is that tracking search engines is nontrivial: either we store the search engines as free-form strings (yuck), or we maintain an enum of all the search engines that we care about.

My intuition is to scale back and record the search term and the timestamp; later, if we require the search engine, we can add a column and bump the db version.

What do you think?

Comment 3

[Richard Newman [:rnewman]]

(In reply to Eric Edens [:eedens] from comment #2)
> Hey Richard, regarding the "which engine" field, I think there are some
> useful things we could with this in the future, but we really haven't
> specced anything that would use it now. (At least with the search activity.)

For Fennec itself, this would replace the results page URL in history, so the search engine is an integral part of that.

The recorded value should be everything you need to get back to the same results.

> Part of my concern is that tracking search engines is nontrivial: either we
> store the search engines as free-form strings (yuck), or we maintain an enum
> of all the search engines that we care about.

For Fennec this should be straightforward: we know the engine you're using for a search -- it's what we use to find out the URL to load.

For the search activity, maybe this concept simply doesn't apply, particularly in a multi-provider world.

How about a schema like this:

  terms            string
  lastVisited      timestamp
  visits           count
  source           "b" | "sa"
  provider         ID, if available
  URL              search URL, if available

The search activity just fills in the first four, with nulls for the last two.

Comment 4

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Richard Newman [:rnewman] from comment #3)

> How about a schema like this:
> 
>   terms            string
>   lastVisited      timestamp
>   visits           count
>   source           "b" | "sa"
>   provider         ID, if available
>   URL              search URL, if available
> 
> The search activity just fills in the first four, with nulls for the last
> two.

I'm wondering if we are over engineering this from a Fennec presepctive. Why would we want the engine and the search URL? Are we thinking that we would re-run the exact same search on the exact same engine? What if the user is using a different default engine now?

In my constant effort to do less work, I wonder why we would save anything but the terms? I imagine we'd consider saving lastVisited and visits (although the "visit" hurts my head) for some sort of frecency. That makes some sense.

I'd push to drop the source, provider and URL though.

Comment 5

[Richard Newman [:rnewman]]

> Why would we want the engine and the search URL?

I'm thinking about this as a replacement for parsing URLs out of history, and indeed as a sibling of history. How would you build a "Recent Searches" home panel?

Storing the engine lets us present an actionable list of terms -- you can go directly back to the search you made (or edit the terms as an optional step).

"The engine" is a composite of ID and URL. Only partner engines have an ID; the rest have a non-unique name, so we track something else instead. And storing the URL is much friendlier to the search activity, which doesn't have easy access to Gecko's search engine list.

Storing just the term would be like storing only the root domain for a page in history -- you could do it, the user could navigate back to where they actually were, but why not store enough data to make it actionable?


> Are we thinking that we would
> re-run the exact same search on the exact same engine?

Yes. This is like a persistent version of Safari's "go back to the search results" feature. Or, from a different perspective: have you ever tried looking for a search you did by browsing through our history pane? What a pain in the ass. Think of this as a much tidier alternative to adding a boolean flag and a few columns to the history table.


> What if the user is
> using a different default engine now?

That's one reason why we keep the engine. If I searched for a word in SpanishDict, and I want to go back to that search, I don't want Fennec to take me to Google.

That is: the place you searched is an integral part of the search.


> I'd push to drop the source, provider and URL though.

I think we've spent more time on this discussion than we would have writing

  cv.put("engine", searchEngine.id);
  cv.put("url", searchURL);

:P

Comment 6

[Nick Alexander :nalexander [he/him]]

Build system note: the way to integrate against the CP being built in Bug 1030277 is to follow android-sync's model.

As a `grunt import` command, import a small set of Fennec's "contract" files into the Search Activity repo.  These don't get mixed into the main src directory; they shouldn't get `grunt export`ed.  Hopefully we can just grab AppConstants.java.in (and preprocess it).  Actually, instead of writing this down, I'm going to write the patches now.

Comment 7

[:Margaret Leibovic]

The core functionality necessary for v1 of the search activity has landed in other bugs, so I'm removing the dependency on our meta bug.

I thought we could just close this bug, but Eric told me that we should keep it open for landing more robust search storage in the future.

Comment 8

[Eric Edens [:eedens]]

Releasing this back into the pool.

I'd be in favor of closing this bug -- when we have more specific features that require additional history support, then we can create bugs that are more focused.