568034 - create event or method to determine the source contentWindow of loading images and media

Status: NEW

(Core :: Graphics: ImageLib, defect)

Description

[David Dahl :ddahl]

It would be very helpful if the contentWindow that loads images and media could be determined when observing http traffic inside an httpActivityObserver.

Here are bz's commens about this from bug 552140:

 Boris Zbarsky (:bz)      2010-05-21 20:58:13 PDT

> It appears that each image or media channel will have a different loadGroup
> from the originating html document that is loaded. (sometimes the loadGroup
> is null)

For images, the necko channel will be in a synthetic loadgroup.  The
imgRequestProxy request will be in the document loadgroup.

Image loads are shared across documents, so you can't really tie them to a
specific document.

I can't speak to what the media loads look like.

Comment 1

[John J. Barton]

Honza, how do you assign images to the net panel in Firebug. Seems like that would work for David as well.

Comment 2

[Jan Honza Odvarko [:Honza] (always need-info? me)]

David, see this code in Firebug's lib.js
http://code.google.com/p/fbug/source/browse/branches/firebug1.6/content/firebug/lib.js#4012

This is the way Firebug use to get parent window for a request and decide whether it belongs to the current page. This doesn't work for e.g. html5 manifest (see #564419) or fav icons since these are not associated with a window.

Honza

Comment 3

[Boris Zbarsky [:bzbarsky]]

Hmm.  I guess imagelib _does_ fake the notification callbacks on the image request itself (while not putting it into the document loadgroup), so that might in fact do the right thing.  But that's the code I told ddahl to use, and he was claiming that it wasn't working for him, no?

Comment 4

[David Dahl :ddahl]

I think the issue is that I am handed an nsIChannel via the httpActivityObserver, and I try to get the nsIRequest via QI. And this condition makes this approach not work.


Here is my latest attempt:

  getLoadContextFromChannel: function HS_getLoadContextFromChannel(aChannel)
  {
    // get a request from the channel
    var request = aChannel.QueryInterface(Ci.nsIRequest);
    try {
      if (request && request.notificationCallbacks) {
        return request.notificationCallbacks.getInterface(Ci.nsILoadContext);
      }
    }
    catch (ex) {
      //noop
    }

    try {
      if (request && request.loadGroup &&
          request.loadGroup.notificationCallbacks) {
        return request.loadGroup.notificationCallbacks.getInterface(Ci.nsILoadContext);
      }
    }
    catch (ex) {
      // noop
    }
    return null;
  },

This always returns null.

Comment 5

[Boris Zbarsky [:bzbarsky]]

>    var request = aChannel.QueryInterface(Ci.nsIRequest);
>    try {
>      if (request && request.notificationCallbacks) {

There is no notificationCallbacks property on nsIRequest.  There _is_ one on nsIChannel, however.  Why do you need the QI to nsIRequest, exactly?  I'm a little surprised XPConnect interface flattening isn't saving you, but maybe that's correct.

So it seems to me that your code should look more like this:

 getLoadContextFromChannel: function HS_getLoadContextFromChannel(aChannel)
 {
   try { 
     return aChannel.notificationCallbacks.getInterface(Ci.nsILoadContext);
   } catch (e) {}
   try {
     return
       aChannel.loadGroup.notificationCallbacks.getInterface(Ci.nsILoadContext);
   } catch (e) {}
   return null;
 }

Comment 6

[David Dahl :ddahl]

The only difference is that I am handed an nsIHttpChannel (inside the nsIHttpActivityObserver), which does not have notificationCallbacks either.

I have also tried to QI to nsIChannel, but no luck, it still returns null.

Comment 7

[Boris Zbarsky [:bzbarsky]]

If you QI to nsIChannel and still have null notificationCallbacks, then you just have null notificationCallbacks on the channel (though this QI shouldn't even be needed for nsIHttpChannel).  This is the case for most channels; just not the ones imagelib opens.

So given that, when do you end up with null returned from this function?  Using what steps to reproduce?  What's the uri of aChannel?

Comment 8

[David Dahl :ddahl]

a uri from one of these channels is:
http://ixiaa.com/tracker.jpg?partner=010&offer=1&goal=1&result=1

STR:

1. I created a nsIHttpActivityObserver,

2. inside of the observeActivity method I filter for ACTIVITY_TYPE_HTTP_TRANSACTION

3. I get the loadGroup from the channel that is provided, which I use to look up in a loadgroups index i keep to match against known windows we are logging.

4. If the loadGroup is null, its URI is almost always an image

5. I attempt to use the channel->loadContext->window->loadGroup method to get the loadGroup.

6. for images it is always null.

If I log the URI.spec I can see all types of content being loaded. (btw: swfs are loaded by a channel that has the same loadgroup as the window.)

Here is the current observer code:

http://pastebin.mozilla.org/729165

on line 33 of this example, the loadContext is always null.

here is the getLoadContextFromChannel method:

  getLoadContextFromChannel: function HS_getLoadContextFromChannel(aChannel)
  {
    try {
      return aChannel.QueryInterface(Ci.nsIChannel).notificationCallbacks.getInterface(Ci.nsILoadContext);
    } catch (e) {}
    try {
      return aChannel.QueryInterface(Ci.nsIChannel).loadGroup.notificationCallbacks.getInterface(Ci.nsILoadContext);
    } catch (e) {}
    return null;
  },

Comment 9

[John J. Barton]

David, I think you are testing for loadGroup first and returning if you don't find it. In our code and Boris' we first try the request (aka channel) directly, and only look for the loadGroup if that fails. For example, the URL you posted we never take the loadGroup code path.

Comment 10

[Boris Zbarsky [:bzbarsky]]

> 4. If the loadGroup is null, its URI is almost always an image

Right.

> on line 33 of this example, the loadContext is always null.

That's really odd.  Have you tried stepping into this code in a debugger?  Specifically, breakpoint in C++ right before your getLoadContextFromChannel call (e.g. set a breakpoint in whatever C++ function your log() calls and log() conditionally on the channel URI so that you know you're hitting an image channel), then breakpoint in nsHttpChannel::GetNotificationCallbacks and see what the concrete type of the callbacks object is?  Is it a nsProgressNotificationProxy?

Comment 11

[David Dahl :ddahl]

(In reply to comment #10)
> 
> > on line 33 of this example, the loadContext is always null.
> 

After reading John's comment, I decided to ignore the loadGroups for now and just try to go through the firebug/bz example code.

The loadContext is indeed there after all for images, not sure about html and script, but those can be handled through the loadGroup. I try to follow through and grab the contentWindow, which works, but trying to get chromeWindow is not always working.

I think I need to write a few tests to help me understand all of the objects here.

Comment 12

[Boris Zbarsky [:bzbarsky]]

> not sure about html and script,

Those will definitely have a loadContext in all cases.

> I think I need to write a few tests to help me understand all of the objects
> here.

Please ask any questions you run into!

Comment 13

[Joe Drew (not getting mail)]

To aid drivers in determining if this should block, could you give some background as to what will be impossible if this doesn't make it?

Comment 14

[David Dahl :ddahl]

(In reply to comment #13)
> To aid drivers in determining if this should block, could you give some
> background as to what will be impossible if this doesn't make it?

Basically, determining the originating contentWindow or tab of each imageRequest is difficult via JS. In which case, the Web Console output does not include imageRequests at all. We would like to only show the current tab's activity in the console.

There is some work in bug 569227 that might make this bug INVALID, but is only available with e10s turned on.

Comment 15

[Bobby Holley (:bholley)]

I don't think I'm realistically going to get to this in time (I've got ~15 blockers right now). Assigning to Joe, since he was the one who arbitrarily assigned it to me.

Comment 16

[David Dahl :ddahl]

(In reply to comment #14)

> There is some work in bug 569227 that might make this bug INVALID, but is only
> available with e10s turned on.

Sadly, the referenced bug will not help us in FX 4. we need a solution for this image loading issue some other way.

Comment 17

[John J. Barton]

David, if you would post a test case here we can see if we can help.  I don't know of any case where the Firebug Net panel misses images.

Comment 18

[David Dahl :ddahl]

(In reply to comment #17)
> David, if you would post a test case here we can see if we can help.  I don't
> know of any case where the Firebug Net panel misses images.

I guess the question is then, does Firebug only show images loading for the currently-focused tab? I was under the impression that Firebug was more global in it's logging.

It is not that we are missing imageLoads, we get all of them, it is that we need to only log the images loaded by the current tab to that tab's WebConsole output.

Worst case scenario, we will only log imageLoads to the currently focused WebConsole. We were hoping to completely segregate logging.

We attempt to get from nsIChannel to nsIDOMWindow in this code:

http://mxr.mozilla.org/mozilla-central/source/toolkit/components/console/hudservice/HUDService.jsm#839

It always fails for images becasue of the ImgRequestProxy

Comment 19

[John J. Barton]

(In reply to comment #18)
> (In reply to comment #17)
> I guess the question is then, does Firebug only show images loading for the
> currently-focused tab? I was under the impression that Firebug was more global
> in it's logging.

Firebug shows all network resources loaded by a page when that page is focused. That is, you don't need to be focused to get the list. Network events come in and they are stored according to the window they belong to. As the user selects different tabs, different stored lists are shown.

Firebug has only per-window views. The only global (application-wide) info that shows up is xpcom errors because these do not have window identification.

Again, a test case will do wonders here.

Comment 20

[Jan Honza Odvarko [:Honza] (always need-info? me)]

> Again, a test case will do wonders here.
+1

Honza

Comment 21

[Joe Drew (not getting mail)]

Now that bug 587573 has landed, is this needed?

Comment 22

[Joe Drew (not getting mail)]

David Dahl tells me this doesn't need to block the release of Fx4.