3.06 KB, text/plain
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 After search for a solution to this problem: http://forums.mozillazine.org/viewtopic.php?t=233973 and speaking with some firefox devs on #developers it would appear that there is no sane method of globally observing all URIs as they are fired by the browser at a stage that will allow you to filter/modify them before they are committed to a particular socket. The aim would be the ability to audit URIs irrespective of protocol (current or future, eg. http,ftp,gopher,irc,feed etc) or origin (ie. user actioned, scripted or embedded) and be able to block, allow or modify them. This will greatly aid development of security, anonymity based extensions aswell as allow easy integration with cgi style content rewriter/translator and webproxy services. Reproducible: Always Steps to Reproduce: There is no API in place to achieve the desired goal.
how much context do you need about the initiator of the URL load? -> core: networking
(In reply to comment #1) > how much context do you need about the initiator of the URL load? > > -> core: networking As much as possible? :) I've only been developing moz extension for a short while, so I'm afraid I can't give you a definitive answer I can get by with little or know parent info... However, it would be handy to have flags to show whether it's a user triggered event (mouse click, addressbar), scripted ( js or css ) or embedded link (image, object) Also a reference to which content window|browser|page|script (or tab?) has it spawed from? Going a step further for markup URIs then perhaps a reference to the tag/node that has the link as an attribute (<a href=..> <img src=..> <iframe src..> <object data=..> etc) - this would make it easy if I wanted to highlight all cross-domain embedded images with red borders for instance.
>However, it would be handy to have flags to show whether it's a user triggered >event (mouse click, addressbar), scripted ( js or css ) or embedded link (image, >object) >Also a reference to which content window|browser|page|script (or tab?) has it >spawed from? try nsIContentPolicy
(In reply to comment #3) > >However, it would be handy to have flags to show whether it's a user triggered > >event (mouse click, addressbar), scripted ( js or css ) or embedded link (image, > >object) > >Also a reference to which content window|browser|page|script (or tab?) has it > >spawed from? > > try nsIContentPolicy Ah yes, thanks. From what I can see nsIContentPolicy is good for completely blocking unwanted content. Some google searching has yielded little in the way of documentation, but it does seem to have been effectively implemented in the Adblock extension. Judging by the somewhat daunting size of the extension's source code it's not exactly straightforward to implement simple URL filtering and seemingly it's not possible to be able to manipulate URIs before sending them on their way (eg. "this request is not allowed, use this content instead"). Without the benefit of thorough testing I'm not sure how easy it is to access contextual information on the source of a URI request, so you can also perform manipulation there (eg replacing blocked images with alert logos at the dom level). Perhaps this is all possible through existing interfaces (I'm still searching for solutions), however it seems to me that even if this is the case the whole process requires a disproportionately large and complicated lump of scripting to achieve even limited functionality. I hope someone can enlighten me to the contrary. (sorry if this is the case, I'm still finding me feet with the framework)
try http://lxr.mozilla.org/seamonkey/source/content/base/public/nsIContentPolicy.idl context is available. > the whole > process requires a disproportionately large and complicated lump of scripting to > achieve even limited functionality. that is not my impression of content policy. as far as I know, all you have to do is add the contractid of your component to the content-policy category, and implement nsIContentPolicy on an object registered for that contractid.
> it's not exactly straightforward to implement simple URL filtering Sure it is. Most of the adblock code is doing things like messing with the page itself (instead of just blocking things). It has a very complicated setup for this (way more complicated than needed), as well as for the multiple different types of filtering it offers, its UI, etc. > and seemingly it's not possible to be able to manipulate URIs before sending With content policy, that's correct. > I'm not sure how easy it is to access contextual information on the source of > a URI request Pretty easy.
Thanks for the pointers everyone! I've spent most of yesterday learning from adblock and playing with content policy, and it looks like it will fit the bill nicely. It's going to take me a while, I suspect, to do my own implementation but at least I can see that it's an attainable goal. As I've often seen in the framework, half the battle seems to be finding the right place to look :) Anyway, going to resolve this one WORKSFORME, unless someone thinks otherwise. Thanks again.
ok, after a brief discussion in #developers, it would seem like a good idea to keep this open, since being able to easily modify URIs would be undeniably useful and could save a lot of unnecessary and not 100% effective work-around code. The question is though would nsIContentPolicy be a good location for this? or is it best handled somewhere in necko (where this bug is filed) Cheers.
nsIContentPolicy doesn't sound like a very good place for this.... Depending on what's desired, overriding the nsIIOService contract may be the way to go, though that would rewrite URIs as the URI objects are being created, not when the channel is opened.
(In reply to comment #9) > nsIContentPolicy doesn't sound like a very good place for this.... > > Depending on what's desired, overriding the nsIIOService contract may be the way > to go, though that would rewrite URIs as the URI objects are being created, not > when the channel is opened I think it would be necessary to intercept them before any channels are opened, otherwise it would be difficult change the nature of the request. (also from an efficiency perspective, if you're wanting to filter large numbers of requests)
(In reply to comment #9) > Depending on what's desired, overriding the nsIIOService contract may be the way > to go, though that would rewrite URIs as the URI objects are being created, not > when the channel is opened. One _could_ make that ioservice override's newChannel method return a special nsIChannel impl that wraps the real channel. care would need to be taken since things expect nsIHttpChannel etc.
Or one could just override the newURI method to change the URI as desired...
(In reply to comment #12) > Or one could just override the newURI method to change the URI as desired... Yes, but that would, I think, change things like the URI in the status bar when hovering over a link, which may or may not be desired.
(In reply to comment #13) > (In reply to comment #12) > > Or one could just override the newURI method to change the URI as desired... > > Yes, but that would, I think, change things like the URI in the status bar when > hovering over a link, which may or may not be desired. As you mentioned to me in #developers, do we still not have the problem that the IOService is often referenced directly by CID within the source code, so substituting functionality via contractid may not work in all cases. (perhaps this doesn't apply to the relevant references though).
(the main reason I'm concerned about that is http://lxr.mozilla.org/seamonkey/source/netwerk/base/public/nsNetUtil.h#90 which references the ioservice by CID)
Hmm... Yeah, that would make things not really work... :(
Plus the IO service is loaded very early on during startup. Even if it were only referenced by ContractID, you'd have a hard time intercepting all consumers. The IO service is initialized by the security system whenever a JS component is loaded, so basically you have to assume that the IO service is already initialized by the time the xpcom-startup notification is dispatched. We could invent an API for watching channels and imposing redirects on those channels. That could be used as a way to redirect requests for various resources to alternate protocols. We could leverage nsIChannelEventSink::REDIRECT_TEMPORARY (or maybe REDIRECT_INTERNAL or some other new flag) to hide the alternate URIs from being shown in the UI. We have to be careful from a security point of view as well. Afterall, such a redirect-based override mechanism would impact JS same-origin checks.
Hi just, giving this old report a nudge. I'm curious if any of the structural changes worked into Deer Park have opened the way to resolving this issue (is there a better way?) Cheers
I'd prefer to see this implimented in the same way as http-on-modify-request, using the observer service. e.g. "channel-on-create-request" A.
Any solution for this yet? Does using nsIIOService to override newChannel or new URI work? I have spent almost 2 weeks digging into mozilla source to find where I need to intercept before channel opens, but it looks like I hit the dead end. Any workaround solutions, advices. -Dark
(In reply to comment #15) > (the main reason I'm concerned about that is > http://lxr.mozilla.org/seamonkey/source/netwerk/base/public/nsNetUtil.h#90 which > references the ioservice by CID) I've just noticed that this issue seems to be resolved with the release of 1.8. IO Service is now being referenced by contractID - if you observe the difference between the old: http://lxr.mozilla.org/mozilla1.7/source/netwerk/base/public/nsNetUtil.h#90 and the new: http://lxr.mozilla.org/seamonkey/source/netwerk/base/public/nsNetUtil.h#96 The issue of timing still remains however. Is there no way we can intercept nsIIOService's contract before it gets initialized?
How about another idea... call the redirection handlers in nsBaseChannel::AsyncOpen (and call nsIChannelEventSink::onRedirect(this, newChannel, REDIRECT_INTERNAL) once all handlers were called), and forward the open to the new channel. That'd have some nice features: - Observers can see the fully-configured channel - Compared to doing it in nsIIOService::newChannelFromURI, it has no reentrancy problems - Compared to doing it in IIOService::newURI it has more flexibility (channel rather than URI), and also no reentrancy problem Downsides: - Relevant channels don't use basechannel yet
further benefits: - newChannel callers get what they expect - newURI callers get what they expect (a URI that represents exactly the passed-in string)
(In reply to comment #22) > How about another idea... call the redirection handlers in > nsBaseChannel::AsyncOpen (and call nsIChannelEventSink::onRedirect(this, > newChannel, REDIRECT_INTERNAL) once all handlers were called), and forward the > open to the new channel. From my somewhat limited knowledge of the framework, this would appear to me to be the most gratifying solution so far. > Downsides: > - Relevant channels don't use basechannel yet > Would seem to be a major stumbling block to me. Although Mr Biesinger has indicated to me that moving existing channels over to the nsBaseChannel implementation is planned for future releases (gecko1.9?). Personally I'd like to see an interim solution that provides at least a limited functionality from ioService:newChannelFromURI() but is designed in such a way to allow a smooth transition to nsBaseChannel::AsyncOpen when the underlying support is there. thanks for everyone's comments thus far.
I like the suggestion of adding observers to anything derived from nsBaseChannel. A generalization of HTTP's on-modify-request and on-examine-response events would also be nice. Another thing we can do I think is add observers inside nsIOService:: NewChannelFromURI. It can call out to consumers and give them the ability to see the newly created channel and substitute that channel for another channel. This is similar to the way you can implement nsIProtocolHandler today and have your NewChannel method return the channel of another protocol handler. I know this means that you don't get to intercept a fully configured channel, but it would still be useful in some cases. Making HTTP and FTP inherit from nsBaseChannel is on my TODO list for Gecko 1.9.
(In reply to comment #25) > Another thing we can do I think is add observers inside nsIOService:: > NewChannelFromURI. It can call out to consumers and give them the ability to > see the newly created channel and substitute that channel for another channel. I want to note that this has re-entrancy problems - if that impl wants to use a different channel, it has to call NewChannelFromURI (usually). So, that would need to be taken care of somehow. > Would seem to be a major stumbling block to me. Well, the code could be done in a helper function that all asyncOpen impls can call.
(In reply to comment #26) > I want to note that this has re-entrancy problems - if that impl wants to use a > different channel, it has to call NewChannelFromURI (usually). So, that would > need to be taken care of somehow. Aye...all I can see to do there is set a flag on re-entry that's used as a conditional for whether to fire the channel redirect handlers. Using a helper function as you suggest would probably make a smoother entry into gecko1.9, hopefully without breaking compatibility for any future 1.8 sourced components that have already made use of channel redirects.
This is important. For me it is the simplest way to permit HTTP filtering by extensions at the network level.
It would be quite a useful thing to get an input/output filter framework inside Mozilla. For example, for certain enhancements to the HTTP protocol I'm working on (OpenPGP Extensions for HTTP, see enigform and/or mod_auth_openpgp in freshmeat), I had to write an apache input filter of CONNECTION level. Apache also provides filters that can work on higher layers, like a simple content filter (transform XML into PDF on the fly, etc). It'd be quite nice for XPCom components to register input (http response) and output (http request) filters in a similar fashion.
I'm working for more than a year on Firekeeper extesnion (http://firekeeper.mozdev.org) that does some advance content filtering (examines request URL, response body, headers and cancels suspicious requests). I must agree that doing it now is really painful. I had to create a lot work around code and still there are some stability problems. Input/output filter would be really useful for extensions like Firekeeper.
Looks like firekeeper does much of what I was planning to do when I opened this bug 2 1/2 years ago (although my goals were for anonymous browsing). I'm still very interested in getting this feature added - my project has been squarely on a back-burner until I can find a competent coder with the time and will to help implement url auditing. I did write a little example code for nsIOService.cpp a couple of years back just to illustrate my idea (I'm not a C++ coder though, so it was never intended as a working example), but I'm guessing much has changed since then and there are probably better ways to implement things. Never-the-less I'll see if I can find it and add it as an attachment for reference sake. Cheers
Created attachment 283918 [details] channel redirect/url rewrite illustrative example implementation Found my example implementation. Just want to stress that this was for illustrative purposes only, and is most likely obsolete. There was an idea a couple of years back to use nsBaseChannel as the point redirect-point, but it looks like it's still not implemented across the board for all protocol handlers (such as http), is this correct?
Please see bug 421224 - I'm wondering if it should be duped to this bug.