1616244 - HAR generated by Netmonitor isn't valid ; "title" key in "pages" section is missing.

Status: RESOLVED FIXED

(DevTools :: Netmonitor, defect, P3)

Description

[lucas]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

User-Agent : Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0

1. Open devtools
2. Load a website
3. Right click "Save all as HAR with content" when the page is loaded
4. Inspect the file

Actual results:

The HAR file generated is missing the "title" key in the "pages" section.
ex :
"pages": [
{
"startedDateTime": "2020-02-18T13:27:29.083+01:00",
"id": "page_1",
"pageTimings": {
"onContentLoad": 2406,
"onLoad": 2991
}
}
],

Expected results:

The "title" key should be present.
The spec mention that this key isn't optional : http://www.softwareishard.com/blog/har-12-spec/#pages .

Comment 1

[Release mgmt bot [:sylvestre / :calixte / :marco for bugbug]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Release mgmt bot [:sylvestre / :calixte / :marco for bugbug]]

The priority flag is not set for this bug.
:Honza, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 3

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Attachment: image.png

Thanks for reporting this!

This can be simply visible in HAR Viewer
http://janodvarko.cz/har/viewer/

Honza

Comment 4

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Looks like this line doesn't produce valid title:

https://searchfox.org/mozilla-central/rev/d5b34cc8872177d3ee071e06f787c2a14268595b/devtools/client/netmonitor/src/har/har-builder.js#83

Honza

Comment 5

[Manish Sahani]

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #4)

Looks like this line doesn't produce valid title:

https://searchfox.org/mozilla-central/rev/d5b34cc8872177d3ee071e06f787c2a14268595b/devtools/client/netmonitor/src/har/har-builder.js#83

Honza

At the buildHarData function in https://searchfox.org/mozilla-central/source/devtools/client/netmonitor/src/har/har-exporter.js is where the title is attached, I tried debugging the title there it turns out to be undefined there also. I think the problem is here at buildHarData.

I would like to work on this.

Comment 6

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thanks for the help, assigned to you.

Honza

Comment 7

[Manish Sahani]

Attachment: Bug 1616244 - Fix invalid HAR generation by netmonitor r=honza

This patch is the fix for invalid HAR generated by netmonitor. The 'title' key was missing
in 'pages' section.

Depends on D65217

Comment 8

[Manish Sahani]

Hey, I have pushed the patch. I have a doubt regarding revisions. This patch is dependent on my last patch, is there something I have to do before committing so that every new patch doesn't depend on the old one?

Comment 9

[Julian Descottes [:jdescottes]]

Since you asked here as well, I'll copy/paste my answer from https://phabricator.services.mozilla.com/D65217#2001705:

You are probably working on top of your previous commit, so when you submit using moz-phab it keeps submitting the whole stack.
try to:

• hg pull central (will fetch the latest changes from mozilla-central)
• hg update central (will move your repository to point to the latest mozilla-central changeset)

Then you'll need to either redo your patch for Bug 1616244, or move your changeset for Bug 1616244 to the tip of mozilla-central.
You can use hg wip to visualize your repository, find your changeset. As an example, if you find that the hash of your changeset is abcdef123456, you can move it using hg rebase: hg rebase -s abcdef123456 -d central

Comment 10

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Manish Sahani from comment #7)

This patch is the fix for invalid HAR generated by netmonitor. The 'title' key was missing
in 'pages' section.

Thanks!

Depends on D65217

Why it depends on D65217. It looks like separate bug.

Btw. I am now seeing:

log.entries[12].response.content.mimeType is missing and it is not optional

With e.g. loading google.com
Can you see that too?

Honza

Comment 11

[Manish Sahani]

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #10)

Btw. I am now seeing:

log.entries[12].response.content.mimeType is missing and it is not optional

With e.g. loading google.com
Can you see that too?

I tried generating log for google.com multiple, content.mimeType is valid for each entry, also there were some other errors with log.entries[1].cache.afterRequest.expires, log.entries[1].cache.afterRequest.lastAccess, log.entries[1].cache.afterRequest.eTag, and log.entries[1].cache.afterRequest.hitCount,

my error log image https://i.ibb.co/mTryGhk/Screenshot-2020-03-09-at-6-33-20-PM.png.

This lines is where the error is https://searchfox.org/mozilla-central/rev/d5b34cc8872177d3ee071e06f787c2a14268595b/devtools/client/netmonitor/src/har/har-builder.js#529.

My doubt is regarding this function buildCacheEntry, are we attaching the actual values at this buildCacheEntry function or is it just a placeholder used for validating the presence of keys as in the current version it returns a boolean (from findKeys function). Should I place the correct values here? and also function does not specify fields for hitCount and lastAccess.

Comment 12

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Manish Sahani from comment #11)

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #10)

I tried generating log for google.com multiple, content.mimeType is valid for each entry, also there were some other errors with log.entries[1].cache.afterRequest.expires, log.entries[1].cache.afterRequest.lastAccess, log.entries[1].cache.afterRequest.eTag, and log.entries[1].cache.afterRequest.hitCount,

This lines is where the error is https://searchfox.org/mozilla-central/rev/d5b34cc8872177d3ee071e06f787c2a14268595b/devtools/client/netmonitor/src/har/har-builder.js#529.

My doubt is regarding this function buildCacheEntry, are we attaching the actual values at this buildCacheEntry function or is it just a placeholder used for validating the presence of keys as in the current version it returns a boolean (from findKeys function). Should I place the correct values here? and also function does not specify fields for hitCount and lastAccess.

I think you are on the right track and yes hitCount and lastAccess should be there too.

It looks like we shouldn't set false as values since string and/or integer is expected in these fields.

Honza

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2055bba77ce3
Fix invalid HAR generation by netmonitor r=Honza

Comment 14

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/2055bba77ce3