1409865 - Investigate validity of wasm/asm.js use counter telemetry

Status: RESOLVED DUPLICATE of bug 1412382

(Core :: JavaScript Engine, enhancement, P3)

Description

[Luke Wagner [:luke]]

Bug 1391633 added use-counter probes for asm.js and wasm.  Looking at the data now for FF57 Beta for USE_COUNTER2_JS_ASMJS_PAGE and USECOUNTER2_JS_WASM_PAGE, it shows 1.74% and 1.48% of page loads using these features, resp.

However, we know that the real usage is much lower using the older JS_AOT_USAGE probe (that these were meant to replace).  We can approximate % of page loads using asm.js/wasm by dividing the Sample Count of JS_AOT_USAGE probe (9.5M) with the Sample Count of some probe that is sampled on every page load like TOTAL_CONTENT_PAGE_LOAD_TIME (12.46B), which gives us .07% of page loads.

Looking at the Sample Count of USE_COUNTERS2_JS_ASMJS_PAGE, I see it is 61.22M which is much less than the 12.46M Sample Count of TOTAL_CONTENT_PAGE_LOAD_TIME over the same period.  In theory, these Sample Counts should be the same, so my best lead is that somehow Sample Counts aren't being counted correctly in the False case.

Looking at other USE_COUNTERS2_*, I see Sample Counts all over the place from USE_COUNTERS2_PROPERTY_FILL_PAGE with 2.91B to USE_COUNTER2_XMLDOCUMENT_ASYNC_setter_PAGE with 6.87M.  So maybe I'm misunderstanding the meaning of Sample Count?

Anyhow, I'd appreciate any help understanding what's the story here.

Comment 1

[Luke Wagner [:luke]]

Francois, maybe you could help?

Comment 2

[Luke Wagner [:luke]]

(Oops, in paragraph 3, I meant 12.46B, not 12.46M, for the Sample Count of TOTAL_CONTENT_PAGE_LOAD_TIME.)

Comment 3

[François Marier [:francois]]

I would suggest asking on #telemetry or on the https://mail.mozilla.org/listinfo/fx-data-dev list.

I know very little about the internals of telemetry.

Comment 4

[Luke Wagner [:luke]]

Nathan, are you familiar with the internals (or know who is)?

Comment 5

[Nathan Froyd [:froydnj]]

(In reply to Luke Wagner [:luke] from comment #0)
> Bug 1391633 added use-counter probes for asm.js and wasm.  Looking at the
> data now for FF57 Beta for USE_COUNTER2_JS_ASMJS_PAGE and
> USECOUNTER2_JS_WASM_PAGE, it shows 1.74% and 1.48% of page loads using these
> features, resp.
> 
> However, we know that the real usage is much lower using the older
> JS_AOT_USAGE probe (that these were meant to replace).  We can approximate %
> of page loads using asm.js/wasm by dividing the Sample Count of JS_AOT_USAGE
> probe (9.5M) with the Sample Count of some probe that is sampled on every
> page load like TOTAL_CONTENT_PAGE_LOAD_TIME (12.46B), which gives us .07% of
> page loads.

That's a pretty big discrepancy!  I don't have any good theories for why this is different, unless we're seeing something like, for an example session of:

1. Visit an asm.js page
2. Visit a different asm.js page
3. Visit a different asm.js page
4. Visit a wasm page

This should accumulate 3 counts for an asm.js counter (or bucket, in the older telemetry world) and 1 count for a wasm counter (bucket).  But does the Sample Count for the histogram correspond to the number of times a given bucket was seen, or the total count of all the samples in the bucket?  The difference between those two measurements for the above session would be 1 vs. 3 for asm.js...but even with differences in counting, it's a long way to get to ~2 orders of magnitude difference...

Oh, maybe the old telemetry usage was only measuring windows, and the new use counter usage is measuring windows *and* workers?  That seems plausible, but...I think we don't collect use counters from workers.  Or at least grepping around, it doesn't look like we would.  So that theory is out the window.

> Looking at the Sample Count of USE_COUNTERS2_JS_ASMJS_PAGE, I see it is
> 61.22M which is much less than the 12.46M Sample Count of
> TOTAL_CONTENT_PAGE_LOAD_TIME over the same period.  In theory, these Sample
> Counts should be the same, so my best lead is that somehow Sample Counts
> aren't being counted correctly in the False case.

I am confused by this.  You're saying that the JS_ASMJS counter should be the same as the number of times we count content page loads?  That seems weird, but that's probably me misunderstanding things!

From a brief glance at TOTAL_CONTENT_PAGE_LOAD_TIME (TCPLT), it looks like TCPLT would get accumulated for *any* document we load, which means for a toplevel page with multiple iframes, we'd get 1 + #iframes measurements on TCPLT.  I'm not sure about my reasoning there, but if that's the case, the number of samples for TCPLT doesn't correspond to a "number of toplevel pages every session we've seen has visited" or something like that.

Georg might have some insight here?

Comment 6

[Luke Wagner [:luke]]

(In reply to Nathan Froyd [:froydnj] from comment #5)
> But does the Sample Count for the histogram correspond to the number of times a given
> bucket was seen, or the total count of all the samples in the bucket?

The old counter simply adds either a 0 (for asm.js) or 1 (for wasm) to the histogram which, iiuc, would mean the sample count is the total number of uses of asm.js or wasm.  And this happens for each *instantiation* of wasm/asm.js, independent of page/document load, so in theory this counter should be *over* counting asm.js/wasm (which was my motivation for switching to a page/document-based use counter in the first place).

> > Looking at the Sample Count of USE_COUNTERS2_JS_ASMJS_PAGE, I see it is
> > 61.22M which is much less than the 12.46M Sample Count of
> > TOTAL_CONTENT_PAGE_LOAD_TIME over the same period.  In theory, these Sample
> > Counts should be the same, so my best lead is that somehow Sample Counts
> > aren't being counted correctly in the False case.
> 
> I am confused by this.  You're saying that the JS_ASMJS counter should be
> the same as the number of times we count content page loads?  That seems
> weird, but that's probably me misunderstanding things!

Looking at the table data, it looks like the reported % is simply taking (count(1) / Sample Count) where Sample Count = count(0) + count(1) and "1" = page/document load where feature was used and "0" = page/document load where feature wasn't used.

So it's based on this that I would expect all the USE_COUNTERS2 to have the exact sample Sample Count and the only thing that varies is the count(0) and count(1) ratios.

But maybe I'm wildly misunderstanding, which seems likely given the huge variance in Sample Counts amount USE_COUNTERS2_* probes; it'd be great to get someone here who knows how this actually works.

> From a brief glance at TOTAL_CONTENT_PAGE_LOAD_TIME (TCPLT), it looks like
> TCPLT would get accumulated for *any* document we load, which means for a
> toplevel page with multiple iframes, we'd get 1 + #iframes measurements on
> TCPLT.  I'm not sure about my reasoning there, but if that's the case, the
> number of samples for TCPLT doesn't correspond to a "number of toplevel
> pages every session we've seen has visited" or something like that.

Ah, I was assuming "PAGE" had the same meaning as "PAGE" in use counters (https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/use-counters.html) which meant only top-level.

Comment 7

[Luke Wagner [:luke]]

(In reply to Nathan Froyd [:froydnj] from comment #5)
> Oh, maybe the old telemetry usage was only measuring windows, and the new
> use counter usage is measuring windows *and* workers?  That seems plausible,
> but...I think we don't collect use counters from workers.  Or at least
> grepping around, it doesn't look like we would.  So that theory is out the
> window.

Good find!  Filed bug 1410468.

Comment 8

[Nathan Froyd [:froydnj]]

(In reply to Luke Wagner [:luke] from comment #7)
> (In reply to Nathan Froyd [:froydnj] from comment #5)
> > Oh, maybe the old telemetry usage was only measuring windows, and the new
> > use counter usage is measuring windows *and* workers?  That seems plausible,
> > but...I think we don't collect use counters from workers.  Or at least
> > grepping around, it doesn't look like we would.  So that theory is out the
> > window.
> 
> Good find!  Filed bug 1410468.

To be clear, use counters aren't collected, period, on workers.

And it seems that if we *were* collecting on workers before and we aren't now, the numbers should be higher before than after.  So I don't think this is a good theory, more of thinking out loud on a bugzilla comment. :)

Comment 9

[Luke Wagner [:luke]]

(In reply to Nathan Froyd [:froydnj] from comment #8)
Agreed with your analysis, I was just thinking that, independent of this bug, we should start collecting this data on workers :)

Comment 10

[Georg Fritzsche [:gfritzsche]]

Before diving dashboard details, just to make sure: did you confirm locally that the expected data is recorded using about:telemetry?

For rough proportional analysis, we have a scalar that counts top-level URIs:
- dashboard: https://mzl.la/2laNMs8
- details: https://georgf.github.io/fx-data-explorer/?search=browser.engagement.total_uri_count&searchtype=in_any&optout=true&channel=nightly&constraint=is_in&version=any

I was comparing the same timeframe for:
- total_uri_count: https://mzl.la/2laNMs8
  -> Ping Count 105.39M
  -> Sample Count 105.39M
- USE_COUNTERS2_JS_ASMJS_PAGE: https://mzl.la/2lbFzDY
  -> Ping Count 651.49k
  -> Sample Count 81.46M

The discrepancy in ping counts might be expected if the use counters are only submitted when the page goes away.
However, i assumed the two measurements should roughly line up for sample count (number of values in all buckets), but that is ~33% off.
I would also assume that the sample counts for all use counters match up as you mentioned.

Is it verified that this works correctly on the client-side?
Do custom kinds of use counters get counted the same way?

Comment 11

[Luke Wagner [:luke]]

Thanks!

So first, I can confirm (with printf()), that nsIDocument::SetChildDocumentUseCounter(eUseCounter_custom_JS_wasm) is called when visiting, e.g., webassembly.org/demo and later that nsIDocument::GetChildDocumentUseCounter(eUseCounter_custom_JS_wasm) is called when loading about:telemetry.  However, on Nightly, I can't find anything in about:telemetry.  On Beta, I could only find JS_AOT_USAGE (the old counter) in about:telemetry -> Raw JSON, but not USE_COUNTERS2_JS_*.  Is there some culling going on?

Next, IIUC, you're saying total_uri_count's Sample Count is the number of *sessions* (which is why Ping Count = Sample Count), which makes sense, given its description.  But then that doesn't seem comparable to USE_COUNTER2_JS_ASMJS_PAGE's Sample Count which I assumed was the number of *page loads*.  If USE_COUNTER2_* Sample Counts were the number of sessions, then the % they report would be the % of *sessions* that contain at least one use of a feature which is a very different statistic.

I have a theory: when a USE_COUNTER2 isn't used at all, it isn't being included in the ping and that is why USE_COUNTERS2_JS_ASMJS_PAGE's Ping Count is so low: some sessions don't use it at all.  This throws off the Sample Count by drastically under-counting the "unused"s.  The short list of USE_COUNTER2_* in the Raw JSON of about:telemetry seems to indicate this is the case.

Comment 12

[Georg Fritzsche [:gfritzsche]]

(In reply to Luke Wagner [:luke] from comment #11)
> I have a theory: when a USE_COUNTER2 isn't used at all, it isn't being
> included in the ping and that is why USE_COUNTERS2_JS_ASMJS_PAGE's Ping
> Count is so low: some sessions don't use it at all.  This throws off the
> Sample Count by drastically under-counting the "unused"s.  The short list of
> USE_COUNTER2_* in the Raw JSON of about:telemetry seems to indicate this is
> the case.

Yes, that is probably the problem. It seems like USE_COUNTERS2_JS_ASMJS_PAGE is not recorded for all pages.
A short-term solution would be to use something like total_uri_count as a base to compare the 1-bucket counts to.


Going a bit more into detail on the above, i think there are four (?) layers here:

(1) The TMO dashboards, which are best ignored for now while we understand the client-side.

(2) Telemetry histograms, which is what use counters end up being recorded into. Specifically they are boolean histograms with true/false counts.
This is what we show in about:telemetry, send out and show in the dashboards.
These are only sent when something is recorded into them (empty/non-existent histograms just mean "nothing was recorded").

(3) Use counter implementation. Each use counter is recorded into a boolean histogram with naming USE_COUNTERS2_<YourCounterName>.
Only when the use counter implementation calls Telemetry::Accumulate() do we display/send/record them.

(4) Usage of the custom use counters. As i understand it, for custom use counters you need to call `SetDocumentAndPageUseCounter()` yourself as appropriate.

I see (4) is covered with [1] & [2]. Is that code called for every page/document?

If not, is the use counters code in (3) actually tracking recording a "not used" for that page/document?
The way this data currently works, i would expect (3) to always record "not used" for all pages/docs and all use counters, unless usage was recorded.

(2) should probably work fine - if JS_AOT_USAGE is successfully recorded, this layer seems to work fine and we should look closer at (3) & (4).

Maybe we should our approach on the use counters (although that's a separate and larger bug).

1: http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/dom/base/UseCounters.conf#96
2: http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/js/xpconnect/src/XPCJSRuntime.cpp#2694

Comment 13

[Luke Wagner [:luke]]

Yay, looks like we're closing in on this!

> I see (4) is covered with [1] & [2]. Is that code called for every page/document?

I confirmed that are only called when asm.js/wasm are used, so not on every page/document.  

> If not, is the use counters code in (3) actually tracking recording a "not used" for that page/document?

I think so: there is an array of bools for each statically-declared feature that is initialized to 'false', so you can always query whether a given feature has been used in this page/document load and get valid results.

I think the error must lie in whatever logic that looks through features to decide what to include in about:telemetry and the ping payload.

Comment 14

[Chris H-C :chutten]

A note in the documentation[1] says:

"The histograms that are generated out of use counters are set to never expire and are opt-in."

Is it possible that you don't have extended telemetry enabled?

[1]: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/use-counters.html

Comment 15

[Georg Fritzsche [:gfritzsche]]

(In reply to Luke Wagner [:luke] from comment #13)
> I think the error must lie in whatever logic that looks through features to
> decide what to include in about:telemetry and the ping payload.

Nathan, do you know about this part? Or know someone who does?

Comment 16

[Nathan Froyd [:froydnj]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #15)
> (In reply to Luke Wagner [:luke] from comment #13)
> > I think the error must lie in whatever logic that looks through features to
> > decide what to include in about:telemetry and the ping payload.
> 
> Nathan, do you know about this part? Or know someone who does?

I do not know anything concerning about:telemetry or who is responsible for about:telemetry. :(

Comment 17

[Georg Fritzsche [:gfritzsche]]

(In reply to Nathan Froyd [:froydnj] from comment #16)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #15)
> > (In reply to Luke Wagner [:luke] from comment #13)
> > > I think the error must lie in whatever logic that looks through features to
> > > decide what to include in about:telemetry and the ping payload.
> > 
> > Nathan, do you know about this part? Or know someone who does?
> 
> I do not know anything concerning about:telemetry or who is responsible for
> about:telemetry. :(

Sorry, that was missing some information.
I assume that Telemetry is working fine and the use counters are not recording this.
Confirming whether or not Telemetry::Accumulate() is called for the USE_COUNTERS2_JS_ASMJS_PAGE etc. should narrow this down.

If JS_AOT_USAGE is recorded successfully, then extended Telemetry is enabled, so that should be fine.

For the record, my team owns Telemetry itself, including about:telemetry [1].

1: https://wiki.mozilla.org/Modules/Toolkit#Telemetry

Comment 18

[Benjamin Bouvier [:bbouvier] (inactive)]

XPC registers a JS callback that calls SetDocumentAndPageUseCounter(obj, use_counters). Going down the rabbit hole, boolean values are set in the different nsIDocument counter arrays:

http://searchfox.org/mozilla-central/source/js/xpconnect/src/XPCJSRuntime.cpp#2691
http://searchfox.org/mozilla-central/source/dom/bindings/BindingUtils.cpp#3735
http://searchfox.org/mozilla-central/source/dom/base/nsIDocument.h#3108
http://searchfox.org/mozilla-central/source/dom/base/nsIDocument.h#3099

I assume Telemetry::Accumulate is called later by the NSIDocument:

http://searchfox.org/mozilla-central/source/dom/base/nsDocument.cpp#13205,13224

Should there be else branches that accumulate 0 if the use counter was false?

Comment 19

[Georg Fritzsche [:gfritzsche]]

Ah, great. The comment a bit further up gives context here:
http://searchfox.org/mozilla-central/rev/dd47bee6468de7e1221b4d006342ad6b9813d0e5/dom/base/nsDocument.cpp#13169

Not accumulating 0 seems intentional for payload size and processing.
Instead the 1s should be seen as proportional to CONTENT_DOCUMENTS_DESTROYED & TOP_LEVEL_CONTENT_DOCUMENTS_DESTROYED.
Short-term this can be done manually or by using e.g. Telemetry.js [1] to build a custom dashboard.
We also need to fix the documentation [2] to make this behavior clear.
Longer-term we should have a proper dashboard or specialized TMO views to avoid this confusion.
@Nathan, does this sound right to you? Is that the expected behavior for all use counters?

What i personally don't understand then is what a 0 accumulation means here, as we do see 0s being recorded: https://mzl.la/2iDf4q9

1: https://github.com/mozilla/telemetry-dashboard/blob/gh-pages/v2/doc.md
2: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/use-counters.html

Comment 20

[Georg Fritzsche [:gfritzsche]]

Instrumenting a local build a bit with logging for the document code path, i see the following happening:
SetUseCounterCallback() for JSUseCounter::WASM
SetDocumentUseCounter() for eUseCounter_custom_JS_wasm, toggling the mUseCounters entry

nsDocument::ReportUseCounters() gets called repeatedly, but it always see value 0 (or unused) for eUseCounter_custom_JS_asmjs & eUseCounter_custom_JS_wasm (both for the top-level document and the normal branch).
http://searchfox.org/mozilla-central/rev/dd47bee6468de7e1221b4d006342ad6b9813d0e5/dom/base/nsDocument.cpp#13192

Am i missing something?

Comment 21

[Georg Fritzsche [:gfritzsche]]

If i got the context in comment 19 right, this hacked dashboard should show the use counters numbers as intended:
http://georgf.github.io/usecounters/index.html

That doesn't solve the issue in comment 20 though.

Comment 22

[Luke Wagner [:luke]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #20)
> nsDocument::ReportUseCounters() gets called repeatedly, but it always see
> value 0 (or unused) for eUseCounter_custom_JS_asmjs &
> eUseCounter_custom_JS_wasm (both for the top-level document and the normal
> branch).

Did you mean | instead of & ?

Comment 23

[Luke Wagner [:luke]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #21)
Thanks! Is this for the release channel?

Also: is there a bug for integrating the fix into the main telemetry dashboard?

Comment 24

[Chris H-C :chutten]

This will not be for the release channel as the release channel is base collection and Use Counters require extended collection.

Comment 25

[Georg Fritzsche [:gfritzsche]]

(In reply to Luke Wagner [:luke] from comment #22)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #20)
> > nsDocument::ReportUseCounters() gets called repeatedly, but it always see
> > value 0 (or unused) for eUseCounter_custom_JS_asmjs &
> > eUseCounter_custom_JS_wasm (both for the top-level document and the normal
> > branch).
> 
> Did you mean | instead of & ?

I mean for both of them i see 0 / unused, i was logging both and loading the tank demo.

(In reply to Luke Wagner [:luke] from comment #23)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #21)
> Thanks! Is this for the release channel?
> 
> Also: is there a bug for integrating the fix into the main telemetry
> dashboard?

That's a quick hack using the beta 57 data.
We don't have useful/representative release channel data for use counters as they are opt-in.

I filed bug 1412382 on the dashboarding.

Comment 26

[Luke Wagner [:luke]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #25)
I can confirm: if I visit webassembly.org/demo and then close, I don't see asm.js/wasm present in any of the calls to nsDocument::ReportUseCounters().

However, if I (1) close the tab after loading webassembly.org/demo, (2) click Minimize Memory Usage in about:memory, then I do see a ReportUseCounters() call reporting 'true'.  So I guess use counter data is harvested from top-level nsDocuments into whatever global accumulator on nsDocument destruction?

Cameron, does the above sound like the expected behavior?

Even when I do this, though, I still don't see asm.js/wasm in the telemetry ping reported by about:telemetry.  I'm guessing this ping is only built periodically.

Is there any way to forcibly re-build the ping payload?

Comment 27

[Georg Fritzsche [:gfritzsche]]

(In reply to Luke Wagner [:luke] from comment #26)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #25)
> I can confirm: if I visit webassembly.org/demo and then close, I don't see
> asm.js/wasm present in any of the calls to nsDocument::ReportUseCounters().
> 
> However, if I (1) close the tab after loading webassembly.org/demo, (2)
> click Minimize Memory Usage in about:memory, then I do see a
> ReportUseCounters() call reporting 'true'.  So I guess use counter data is
> harvested from top-level nsDocuments into whatever global accumulator on
> nsDocument destruction?

Great, that makes sense. Looking at the callers of nsDocument::ReportUseCounters(), nsDocument::~nsDocument() looks like the main one:
http://searchfox.org/mozilla-central/search?q=symbol:_ZN10nsDocument17ReportUseCountersENS_20UseCounterReportKindE&redirect=false

> Even when I do this, though, I still don't see asm.js/wasm in the telemetry
> ping reported by about:telemetry.  I'm guessing this ping is only built
> periodically.

The ping on about:telemetry is built each time you reload the page.
On Nightly i see the JS_ASM/WASM counters with these steps (load demo, close tab, minimize memory usage, reload about:telemetry).

With a recent local dev build you might be affected by bug 1411269 here (dev builds don't record the "extended" Telemetry by default).
Use the pref toolkit.telemetry.testing.overridePreRelease;true to override this. The about:telemetry start page should say "Telemetry is collecting pre-release data".

Comment 28

[Georg Fritzsche [:gfritzsche]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #27)
> (In reply to Luke Wagner [:luke] from comment #26)
> > (In reply to Georg Fritzsche [:gfritzsche] from comment #25)
> > I can confirm: if I visit webassembly.org/demo and then close, I don't see
> > asm.js/wasm present in any of the calls to nsDocument::ReportUseCounters().
> > 
> > However, if I (1) close the tab after loading webassembly.org/demo, (2)
> > click Minimize Memory Usage in about:memory, then I do see a
> > ReportUseCounters() call reporting 'true'.  So I guess use counter data is
> > harvested from top-level nsDocuments into whatever global accumulator on
> > nsDocument destruction?
> 
> Great, that makes sense. Looking at the callers of
> nsDocument::ReportUseCounters(), nsDocument::~nsDocument() looks like the
> main one:
> http://searchfox.org/mozilla-central/search?q=symbol:
> _ZN10nsDocument17ReportUseCountersENS_20UseCounterReportKindE&redirect=false

One possible concern here: Will these actually be submitted sometime during Firefox shutdown?
My quick logging last week didn't show any ReportUseCounter() calls with true during shutdown.

Comment 29

[Luke Wagner [:luke]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #27)
> With a recent local dev build you might be affected by bug 1411269 here (dev
> builds don't record the "extended" Telemetry by default).

Ah hah, that explains it.

(In reply to Georg Fritzsche [:gfritzsche] from comment #28)
> One possible concern here: Will these actually be submitted sometime during
> Firefox shutdown?
> My quick logging last week didn't show any ReportUseCounter() calls with
> true during shutdown.

I was wondering the same thing because I was observing nothing happening when I closed the tab with the demo open.  It seems like we're just assuming that the user's browser is open for a long enough time that losing the last few nsDocuments in the session is no big deal.

Comment 30

[Georg Fritzsche [:gfritzsche]]

(In reply to Luke Wagner [:luke] from comment #29)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #28)
> > One possible concern here: Will these actually be submitted sometime during
> > Firefox shutdown?
> > My quick logging last week didn't show any ReportUseCounter() calls with
> > true during shutdown.
> 
> I was wondering the same thing because I was observing nothing happening
> when I closed the tab with the demo open.  It seems like we're just assuming
> that the user's browser is open for a long enough time that losing the last
> few nsDocuments in the session is no big deal.

That might undercount some important use cases. Ideally use counters would be submitted from "profile-before-change" at the latest.
Should we move this to a separate bug?

Comment 31

[Luke Wagner [:luke]]

Yes, please feel free to file that bug.

With that bug and bug 1412382 filed, and the temporary dashboard http://georgf.github.io/usecounters/index.html, I think this bug can be closed.  I'll make it a dup of bug 1412382 since that's the main problem.  Thanks so much for all your help!

Comment 32

[Georg Fritzsche [:gfritzsche]]

I opened bug 1413266 on investigating the shutdown.

Comment 33

[Luke Wagner [:luke]]

Recently Chrome landed a change to their use counter machinery which caused a large change in the reported percentages.  This apparently was the result in some bugfixes that over-counted #-PageViews (and thus under-reported percentages):
  https://groups.google.com/a/chromium.org/forum/#!msg/blink-api-owners-discuss/IpIkbz0qtrM/HUCfSMv2AQAJ
As a result, "use asm" is now reported as .43% of page views:
  https://www.chromestatus.com/metrics/feature/timeline/popularity/473
I wonder if any of the same logical bugs (particularly the one about mis-counting SVG images as separate page loads) apply to us.