1427502 - [meta] enhancements for for new tab extensions

Status: RESOLVED INCOMPLETE

(WebExtensions :: General, enhancement)

Description

[Sören Hentzschel]

There are still several restrictions with the WebExtensions Framework for new tab extensions. I already opened a few tickets in the past and commented in other tickets regarding APIs needed by New Tab Override (https://addons.mozilla.org/en-US/firefox/addon/new-tab-override/).

Since I still have to use workarounds for some things, don't see solution for other things and some things were denied I opened this ticket to ASK YOU, Mozilla developers, to make suggestions which improvements can be made because I really don't know what exactly I should / can suggest.

My goal is to get at least some improvements for the next ESR release of Firefox (Firefox 60). New tab extensions are used by a few hundred thousand (!) users so it's not a small target audience.

It's kind of a meta ticket / design discussion with you developers and I hope together we can find solutions to improve the experience of new tab extensions. I think a clear overview about the current state is needed for you to understand the current limitations and expectations for add-on developers and the users of these add-ons.

Let me start with the core part of current implementation of New Tab Override. Then I will explain the most important requirements, current solutions and restrictions.

CURRENT CODE OF NEW TAB OVERRIDE:
=======

async openNewTabPage (url, focus_website) {
    await browser.tabs.getCurrent((tab) => {
      // set focus on website
      if (focus_website) {
        // we need to pass the cookieStoreId to support the container tabs feature of Firefox
        browser.tabs.create({ url : url || 'about:blank', cookieStoreId : tab.cookieStoreId }, () => {
          browser.tabs.remove(tab.id);
        });
      }
      // set focus on address bar
      else {
        // we explicitly set the tab id of the current tab to support the edge case of opening a new tab in the
        // background, for support of add-ons like Gesturefy; we set loadReplace to true to disable the back button
        browser.tabs.update(tab.id, { url : url || 'about:blank', loadReplace : true }, () => {
          // there is nothing to do, but it's needed, otherwise browser.history.deleteUrl() does not work
        });
      }
    });

    // delete spammy new tab page entry from history
    browser.history.deleteUrl({ url : browser.extension.getURL(NEW_TAB_PAGE) });
  }
};

REQUIREMENTS:
=======

These requirements are already solved:

1. The user can set the new tab page to any page they want. It works for any http/https page, not for file:// URIs, but the access to local files is a separate issue and should not be discussed as part of this ticket.

2. The back button has to be disabled after loading the desired new tab page. Works (thanks to loadReplace parameter).

3. There shouldn't be a conflict with the container tabs feature of Firefox. There can be a conflict when using the option to set the focus to the website because I have to use a ugly workaround (see paragraph "improvement wanted") but it works with the cookieStoreId parameter.

improvement wanted:
----

1. the user is able to set the focus either to the address bar or to the website (for example the Google search field). Both is possible but to set the focus to the website a really ugly workaround is needed: a new tab has to be created and the current tab has to be closed. It's not obvious at all for developers that creating and updating a tab results in a different focus behaviour. It would be nice if a property could be added to the create() and update() methods to define the focus behaviour.

But it's not only about having a ugly workaround, the current solution also has downsides (besides unnecessary complex code because I need two different code pathes only because of this): the creating of a new tab and closing of the current tab causes a blank tab in the "recently closed tabs", even if I delete the URL from history and that's a problem for the user experience.

Please understand that both (!) focus behaviours are wanted by a lot of users and one of the most important requirements for new tab extensions, not only New Tab Override.

2. It's not acceptable to have moz-extension:// URIs in the history. It's already possible to delete these entries from history but a permission is needed to access the full browser history only to remove the own add-on page from history. But the add-on page does not make sense at all for new tab extensions. So what we add-on developers really need is an option to prevent this pages from appearing in the history without the need to request so much control because we don't want to manipulate the history and it's a problem for the user's trust in our extensions if a new tab extension requires access to the history.

Also, see 1: empty pages still appears in the "recently closed tab" list, that's also bad for the user experience.

And another workaround is needed here: I have to use the following code with an empty (!) block inside:

browser.tabs.update(tab.id, { url : url || 'about:blank', loadReplace : true }, () => {
  // there is nothing to do, but it's needed, otherwise browser.history.deleteUrl() does not work
});

… because otherwise browser.history.deleteUrl() does not work after:

browser.tabs.update(tab.id, { url : url || 'about:blank', loadReplace : true });

That's not obvious for developers. It caused a lot of frustration until I understood that because it was not clear and not documented.

not possible at all:
----

1. Clearing the address bar. You already allowed this via manifest overrides but that's useless for all new tab extensions which let the user decide about the new tab page. Plase understand that this is another really important part for users of these add-ons. A lot of users of add-ons like New Tab Override want an empty address bar on new tab pages.

But even if you don't allow this then it should be selected - that's a MINIMUM REQUIREMENT because, really, add-ons like New Tab Override are for the trash if you have to manually empty the address bar on every (!) new tab because you can't start typing (that's bug 1372996).


These are only the most important requirements. New Tab Override has more requirements like loading local files or overriding browser.display.background_color (that's bug 1372996) but solving the issues mentioned in this ticket has the highest priority because these issues are about the core functionality and the experience for developers and users. Without solving these issues I will never be able to call New Tab Override a great extension, these limitations are too bad.

Comment 1

[Sören Hentzschel]

(sorry, I got confused with the bug numbers. the selecting the address bar ticket is bug 1409675 and the browser.display.background_color ticket is bug 1427071 but it's not important for this ticket)

Comment 2

[Andrew Swan [:aswan]]

Having a bunch of different requests in one bug isn't very practical for making progress.  The usual practice here is to create separate bugs for each individual request and mark them all as blocking this bug.  Note though that some of your requests already have existing bugs (eg bug 1295400)

Comment 3

[Sören Hentzschel]

(In reply to Andrew Swan [:aswan] from comment #2)
> Having a bunch of different requests in one bug isn't very practical for
> making progress.  The usual practice here is to create separate bugs for
> each individual request

I know the usual practice. But the usual practice requires that there are concrete proposals for each individual enhancement. I don't have these proposals. ;) I tried the way of creating new tickets and it was not successful. In my opinion one reason for that is that new tab extensions have many requirements and we have to see the full picture to really understand the requirements of new tab extensions for developers and users.

So I opened this ticket because I think going one step back is the right way *to make progress*: instead of filing new bugs and repeat the arguments over and over again let's summarize all the requirements and see how we can enhance the existing APIs to resolve all or at least most of these requirements. Maybe it's important to know all requirements to decide how the improved APIs could look like.

Yes, it's not the usual way. But maybe this unusual approach helps to make progress. :)

Comment 4

[Andrew Swan [:aswan]]

(In reply to Sören Hentzschel from comment #3)
> I know the usual practice. But the usual practice requires that there are
> concrete proposals for each individual enhancement. I don't have these
> proposals. ;) I tried the way of creating new tickets and it was not
> successful.

I don't understand, you say you don't have a list of the individual enhancements you want but also that you previously created individual bugs for them?  In any case, if there are existing bugs for enhancements you need, please link them here (by making this bug depend on them).  Regardless, trying to have a discussion about a collection of loosely related features on a single bug is difficult and is much *less* likely to lead to any concrete action than having separate bugs for your individual requests.

> In my opinion one reason for that is that new tab extensions
> have many requirements and we have to see the full picture to really
> understand the requirements of new tab extensions for developers and users.

Feel free to link from other bugs to the lengthy description you posted here or to other comments that are relevant.

Comment 5

[Tim Nguyen :ntim]

Looking at the high-level issue, a new tab page set by New Tab Override doesn't behave like one, because New Tab Override sets the NTP to an extension page, then redirects to the actual page set by the user. Is that correct ?

If that is so, I think one way to solve the problem is to allow setting the new tab override through browserSettings (and also allow remote URLs to be set as NTP). This way, instead of setting the NTP to an extension page then redirecting to the wanted page, the extension can just use browserSettings to set the NTP directly, and the set NTP would actually behave like one because there is no redirect.

Alternatively, if exposing this to browserSettings isn't wanted, then New Tab Override could become an online tool where people would be able to create their own NTP add-on with chrome_url_overrides.newtab set to their favorite page. But that also assumes that remote URLs would be allowed in chrome_url_overrides.newtab.

This seems like a high-level solution to solve the New Tab Override issues pointed out above.

> 2. It's not acceptable to have moz-extension:// URIs in the history. It's already possible to delete these entries from history but a permission is needed to access the full browser history only to remove the own add-on page from history. But the add-on page does not make sense at all for new tab extensions. So what we add-on developers really need is an option to prevent this pages from appearing in the history without the need to request so much control because we don't want to manipulate the history and it's a problem for the user's trust in our extensions if a new tab extension requires access to the history.

Adding an entry in the history is actually consistent with Firefox native behaviour: if you navigate away from about:newtab, you will get a history entry.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The meta keyword is there, the bug doesn't depend on other bugs and there is no activity for 12 months.
:ddurst, maybe it's time to close this bug?

Comment 7

[David Durst [:ddurst]]

I'm going to assume this is superseded by other, more specific bugs.