1226547 - (webext-port-abp) [tracking] Support Adblock Plus as a WebExtension

Status: RESOLVED FIXED

(WebExtensions :: Compatibility, defect, P3)

Description

[Erik Vold]

Kev Needham (https://blog.mozilla.org/addons/2015/08/21/the-future-of-developing-firefox-add-ons/) and Bill McCloskey (https://billmccloskey.wordpress.com/2015/08/21/firefox-add-on-changes/) have stated that Mozilla wants to support existing add-ons and not abandon support for existing popular add-ons.

Adblock Plus for Firefox is much more powerful than it's counterpart for Google Chrome, so simply converting the Google chrome version to Firefox is not going to achieve the "support" that Bill and Kev talk about.

It would be nice if someone above would describe how webextensions are going to support this add-on, an how they think we should go about using webextensions, especially without any way to run tests for add-ons like jpm provided for jetpacks and old-school add-ons.

I don't think saying it will "use the native.js idea" is good enough.  This bug will need some real information.

Comment 1

[Giorgio Maone [:ma1]]

(In reply to Erik Vold from comment #0)

> Adblock Plus for Firefox is much more powerful than it's counterpart for
> Google Chrome

A good starting point would be a list of the "more powerful" things Adblock Plus does on Firefox which are not supported by the Chrome Extensions API. Could someone (you?) add it here?

The main APIs necessary to do it, however, are certainly chrome.webRequest and CSS injection (chrome.tabs.insertCSS?). The latter likely needs to come with a "super-power" over its Chrome counterpart, which gives inserted stylesheets lower priority than content and does not honor !important: we should have an "overrideContent" flag or something like that, in order to prevent web pages from trumping extension-provided CSS rules.

> I don't think saying it will "use the native.js idea" is good enough.  This
> bug will need some real information.

I'm adding the most obvious blockers, thanks for opening this tracker.

Comment 2

[Wladimir Palant]

Bug 1215035 exists for the very reason that I don't know what will be missing - that will have to be tested.

Bug 1210990 is absolutely essential of course. I am not sure whether this one will solve the problem of getting the frame hierarchy reliably or whether the content script will be unable to access it like in Chrome. If it is the latter then it is essential that all frames are seen by the extension (including about:blank, javascript: and data: frames) - otherwise the webRequest's parentFrameId member won't be very useful. Also, it's important that the content scripts indeed run in all frames, meaning that content scripts are injected by the principal of a page rather than its URL (about:blank frame inherits principal from parent).

Then it is chrome.tabs.insertCSS() indeed, to be applied to individual frames as proper user script (rules with !important not overridable by content). The solution should not regress bug 77999 - same CSS rules in 20 tabs shouldn't mean 20 times the memory usage. It's also important that it is possible to detect which rules actually match, something that is currently only possible in Firefox (via a hack: -moz-binding with a custom protocol handler).

Finally, there is our list of blockable items that we display at the bottom of the browser - there should be a way to create a UI widget like this and open it programmatically (meaning: when the user selects the corresponding option from our menu). Displaying it as a sidebar isn't really an option because of the size limitations, displaying it as a Developer Tools panel would work however.

That's what I can come up with right now, so far I didn't even start trying it out however.

Comment 3

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

I think this is the list we compiled recently (from Sebastian Noack):

* insertCSS() on Firefox will support an option to to insert a user (instead author) stylesheet, as well as an option to address a specific frameId. Moreover, identical stylesheets inserted for different tabs/frames will be shared across documents.
* In order to associate DOM elements to requests, there will be a DOM event, triggered in the content script, before any webRequest event is triggered in the background page. The content script can add data to the event which will be passed to the webRequest event handlers in the background page. Vice-versa, the event handlers in the background page can add data that will be passed to another event triggered in the content script once the request has been processed by the background page.
* There will be a way to create windows without browser UI, in order to implement dialogs like the one for Element Hiding Helper, blockable items, etc. These however obviously will use HTML instead of XUL.
* In order to reliably highlight elements to be blocked by Element Hiding Helper or to show the tab on embedded Flash and Java elements, a way to manipulate the page content invisible to the JS and CSS of the web page itself, is necessary. It seems that private shadow DOM should do the trick.
* Tracking element hiding hits without XBL bindings is still something we didn't come up with a good solution for. We might have to give up on that.

Beyond this, we'll need to improve our compatibility with Chrome's API.

Comment 4

[Wladimir Palant]

(In reply to Bill McCloskey (:billm) from comment #3)
> * Tracking element hiding hits without XBL bindings is still something we
> didn't come up with a good solution for. We might have to give up on that.

Being able to track hits is the main reason all our filter list authors use Firefox and not Chrome - without that their work is extremely complicated. We've looked for ways to drop the hacks here but even running all selectors through document.querySelectorAll() on demand (yes, extremely slow) isn't going to solve all the use cases.

Comment 5

[Erik Vold]

(In reply to Wladimir Palant from comment #4)
> (In reply to Bill McCloskey (:billm) from comment #3)
> > * Tracking element hiding hits without XBL bindings is still something we
> > didn't come up with a good solution for. We might have to give up on that.
> 
> Being able to track hits is the main reason all our filter list authors use
> Firefox and not Chrome - without that their work is extremely complicated.
> We've looked for ways to drop the hacks here but even running all selectors
> through document.querySelectorAll() on demand (yes, extremely slow) isn't
> going to solve all the use cases.

could this xbl binding ship with fx and just report stats back to us?

Comment 6

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

I think the filter list authors want instant feedback on this stuff. I don't know though.

I don't really understand all the issues here though. The current version of Adblock is actually capable of making blocking decisions for each CSS rule match (i.e., it calls processNode when a rule matches and decides whether to hide the element based on the result). I'm not sure if it needs that flexibility or not. I also don't understand why the querySelectorAll solution wouldn't be sufficient for counting filter hits.

Somehow I'd really like to move us away from the XBL method if possible. It's really inefficient to be running so much JS code for every CSS match.

Anyway, Wladimir is the right person to answer these questions.

Comment 7

[Wladimir Palant]

(In reply to Bill McCloskey (:billm) from comment #6)
> I also don't understand why the querySelectorAll
> solution wouldn't be sufficient for counting filter hits.

I don't like the XBL hack either, but documents are dynamic. By the time we run querySelectorAll the relevant element might already be gone or it might not be added yet (the former being particularly problematic when the website uses a temporary element to detect Adblock Plus). This makes it hard for filter authors to see the impact of their filters reliably. Mutation observers might help here to some degree, if one is willing to query 40k selectors each time an element is added to the document. In general, the performance impact of that solution would be prohibitive - using it to count hits wouldn't really be an option, only inspecting the current page when the list of blockable items or issue reporter are opened. Without filter hit counts pruning the filter lists will become pretty impossible however.

Comment 8

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

What about an API to count how many times a CSS rules triggered on a given page? I don't know how feasible that is, but it might be possible. It would still be necessary to map CSS rules to filters, but that seems possible. Maybe we could even provide a way to tag CSS rules uniquely.

Comment 9

[Wladimir Palant]

(In reply to Bill McCloskey (:billm) from comment #8)
> What about an API to count how many times a CSS rules triggered on a given
> page?

Yes, that's what we need - getting some kind of notification if a CSS rule is triggered, can be async. How many times doesn't really matter, right now we only get one hit per page already no matter how many elements are affected.

Comment 10

[Erik Vold]

(In reply to Wladimir Palant from comment #9)
> (In reply to Bill McCloskey (:billm) from comment #8)
> > What about an API to count how many times a CSS rules triggered on a given
> > page?
> 
> Yes, that's what we need - getting some kind of notification if a CSS rule
> is triggered, can be async. How many times doesn't really matter, right now
> we only get one hit per page already no matter how many elements are
> affected.

I have to say this api sounds useful for more than just abp, and it could be faster than the xbl method abp uses now.  And since it sounds like the biggest issue it is probably worth doing.

In regards to extending Fx's capabilities in general:

I've kind of assumed that xbl would be removed at some point.  It is powerful, but also hard to use and often has some alternative.  Jetpack didn't have any solution for this problem, it kicked the problem down the road further though, and bought time for a solution.

We should start thinking about what the solution is when xbl is required, or something else very powerful (ie where webextensions/jetpack both run into problems when porting old school add-ons).

Creating apis as needed that ship with fx is the ultimate solution for all needs imo.  So I think the best thing to do is to make it easier for third party developers to add apis (ie create an api, and make them opt-in for add-ons so that they are not exposed to all add-ons/webextensions by default).

Basically make firefox release the npm of jetpack.

So I think there are two ways to go about this:

1) build the apis that are needed now adhoc quickly, and figure out how to make third contribution easy later (or never)
2) make it easy for add-on developers to add apis to fx, easy to write tests, and so on from the start and encourage apis from third parties.

I could make some suggestions for #2 if desired.

Comment 11

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

I'm not sure what you mean Erik. Finding out if a CSS rule triggered will require support at the platform level, deep in the CSS code. It's not something we want add-ons to be able to do.

In general, native.js is supposed to be the mechanism for developers to create APIs for add-ons, but that will only be for things already exposed to JS through XPCOM/XUL/XBL. The hope is that, as we move away from XBL or whatever, we'll only need to update the native.js extensions and not all add-ons. How we update them will depend on what we replace XBL with, though, and that hasn't been decided afaik. WebComponents is the obvious solution, but I think there are a lot of problems with that.

Comment 12

[Erik Vold]

(In reply to Bill McCloskey (:billm) from comment #11)
> I'm not sure what you mean Erik. Finding out if a CSS rule triggered will
> require support at the platform level, deep in the CSS code. It's not
> something we want add-ons to be able to do.

Yes, so the platform team will need to be more involved to develop these apis afaict, and add-on devs will need better ways of requesting and even helping to build these deeper integrations.  Unless some alternative plan is developed.

> In general, native.js is supposed to be the mechanism for developers to
> create APIs for add-ons, but that will only be for things already exposed to
> JS through XPCOM/XUL/XBL. The hope is that, as we move away from XBL or
> whatever, we'll only need to update the native.js extensions and not all
> add-ons. How we update them will depend on what we replace XBL with, though,
> and that hasn't been decided afaik. WebComponents is the obvious solution,
> but I think there are a lot of problems with that.

Well I think the Webextensions plan is going to force those working on it to come up with the plan for replacing xbl now, whereas the jetpack plan would've allowed Mozilla to wait until servo actually ships.

I think the native.js approach has a few issues but I'll leave that debate for another thread.  If this is the approach that you want to take ultimately, then add-on developers will need it implemented asap.

Comment 13

[Wladimir Palant]

(In reply to Bill McCloskey (:billm) from comment #11)
> I'm not sure what you mean Erik. Finding out if a CSS rule triggered will
> require support at the platform level, deep in the CSS code. It's not
> something we want add-ons to be able to do.

Maybe not too deep. XBL isn't really necessary, it is sufficient that the CSS rule match sets off a request that can be received by the extension. It could be something like the following:

  #foo {
    display: none;
    background-image: url(moz-extension://.../ping?12345);
  }

And the existing moz-extension protocol handler could then convert this request into an async message to the extension. I'm not sure whether this will work however, last I checked (years ago) background-image was cached and would only produce a request for the first match - with the stylesheet being shared across multiple documents this is an issue. That's why we went with -moz-binding, it wasn't cached.

Comment 16

[Andy McKay]

There's a couple of bugs on here that are about improving ABP support on Firefox and not getting parity to Chrome on Android and Desktop. Specifically bug 1210990, bug 1310037 and bug 1305237. I know we've had a few ABP bugs, but I wouldn't mind it if we could move anything Chrome doesn't do into another tracking bug so we can be sure we've got the minimum in place.

Comment 17

[Wladimir Palant]

(In reply to Andy McKay [:andym] from comment #16)
> There's a couple of bugs on here that are about improving ABP support on
> Firefox and not getting parity to Chrome on Android and Desktop.
> Specifically bug 1210990, bug 1310037 and bug 1305237. I know we've had a
> few ABP bugs, but I wouldn't mind it if we could move anything Chrome
> doesn't do into another tracking bug so we can be sure we've got the minimum
> in place.

Quite frankly, the important parity target for Adblock Plus is not Adblock Plus for Chrome (which has lots of issues and limitations) but rather the current Adblock Plus for Firefox.

Comment 19

[pawel]

For my strange Why Adblock Plus for Firefox 2.99.0.1866beta use the same filter for Polish website doesn't block it as uBlock Origin by Raymond Hill which use the same filter. 

https://www.certyficate.it/polski-filtr-adblock/

Button block some Ad still not works ??

Comment 20

[pawel]

Włądimir message for you .And question WHY ???

Adblock Plus has lost 5 million users



https://www.ghacks.net/2017/09/29/firefox-adblock-plus-lost-millions-of-users-in-the-past-year/

Comment 21

[pawel]

Władimir Polish website doesn't works with your new beta ABB plus .please also implement Polish language for new ABP plus Beta .

https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/versions/beta

Comment 22

[pawel]

My ask about  """"""""""POLISH WEBSITE """""" answered from forum 

##########################################################
@kolor
It seems you didn't read:
https://adblockplus.org/forum/viewtopic.php?f=1&t=52490

As I said: all those sites in polish language are fixable only using advanced features you can find in uBo. ABP didn't implement such advanced methods.
And those sites are already fixed in that polish list (using unknown syntax for ABP).

@smed79 I doubt the maintainers of that list will work again only to produce filters for ABP.

###########################################################

Comment 23

[Wladimir Palant]

Pawel, this is a tracking bug. Adding noise to it isn't going to achieve anything, neither does discussing Adblock Plus requests in a Firefox bug tracker. Mailing me directly won't work either by the way, as I explained to you previously I am only one developer and the time I spend on Adblock Plus is very limited. The Adblock Plus issue tracker is here: https://issues.adblockplus.org/

Comment 24

[pawel]

Władimir I understand you completely.I did some ticket on this list nobody answered .Any from support too.
Noticed In Poland people complain and choose uBlock Original .We don't have change and chosse to do .Or you can don't use uBlock Original and see all times advert.
Poland has the first % place country which block advert in the World.You lose users !


https://issues.adblockplus.org/ticket/5962

Comment 25

[pawel]

Any help for Polish users

https://issues.adblockplus.org/ticket/6057#ticket

Comment 26

[Gregory Pappas [:gregp]]

Adblock Plus is listed on addons.mozilla.org and seems actively maintained.