Client needs to report number of shared URLs on Desktop

VERIFIED FIXED in Firefox 34

Status

defect
P1
normal
VERIFIED FIXED
5 years ago
5 years ago

People

(Reporter: RT, Assigned: Paolo)

Tracking

unspecified
mozilla35
Points:
2
Dependency tree / graph
Bug Flags:
firefox-backlog +
in-moztrap -
qe-verify +

Firefox Tracking Flags

(firefox34 verified, firefox35 verified)

Details

(Whiteboard: [loop-uplift])

User Story

As a product manager I want to know daily the number of shared URLs on FFOS and Firefox clients so that I know how often users try to communicate.

Rough outline of work:

- Add several new items in toolkit/components/telemetry/Histograms.json
-- One for the count of urls generated, then others for the values in comment 3.
- In MozLoopAPI's noteCallUrlExpiry, update the histogram for the count of urls generated: not sure if this is a simple add, or if we need to keep an extra local count.
- Add a new function in MozLoopAPI that is called whenever a url is copied and updates the telemetry histogram.

xref patches on bug 1016138, bug 1014957 and others

Attachments

(1 attachment, 1 obsolete attachment)

Reporter

Description

5 years ago
No description provided.
Reporter

Updated

5 years ago
User Story: (updated)

Comment 1

5 years ago
more info:  what info are we hoping to gain - on the generator side of creating a link or clicker side - and is this that valuable since we are moving past this to FxA as soon as possible.
Flags: needinfo?(rtestard)
Priority: P1 → P3
Reporter

Comment 2

5 years ago
This is intended to gain an understanding of whether people share URLs (copied to clipboard or shared by email) in order to work out if the URL sharing solution is appealing to users.

On the link clicker side we have https://bugzilla.mozilla.org/show_bug.cgi?id=974904

If people like the link sharing mechanism and the account-less mode we should keep it beyond MVP.
Flags: needinfo?(rtestard)
Reporter

Updated

5 years ago
Priority: P3 → P1
Reporter

Comment 3

5 years ago
We need to report this per user agent (i.e Windows, Linux, Mac, FFOS, ...).

On all desktop platforms we need to track the following actions and we will assume that they resulted in shared URLs (we'll assume in post analysis that a percentage of these did not end-up being shared even if they were copied):
* Select and copy (Ctrl-C)
* Copy button
* Share button (opens-up the e-mail composer)

We'll have to assume some people copy but never share - there will be a noise percentage to assume in our statistics.
Reporter

Comment 4

5 years ago
Please note that after discussions with TEF, we need this statistic both for Desktop Loop users and FFOS Loop users in order to be able to differentiate URLs shared from both platforms.
Please confirm if you will have the client identifier necessary to provide this.
User Story: (updated)
If we want the information in comment 3, then the simplest way is we either have to hook into telemetry, or Firefox Health Report. These would not have 100% user coverage. Anything else would require a separate request to the user for allowing us to send this anonymous feedback.

I can't speak to FFOS here, but that should be a separate bug anyway, as I suspect they won't be hooking into the same systems.
Reporter

Comment 6

5 years ago
Katie, we're coming to this one soon on the client side.
Can you please provide us some details on the best way to store the data on the servers?
Flags: needinfo?(kparlante)

Comment 7

5 years ago
(In reply to Romain Testard [:RT] from comment #6)
> Katie, we're coming to this one soon on the client side.
> Can you please provide us some details on the best way to store the data on
> the servers?

If the data is being collected via telemetry or firefox health reports, then the data will be stored by either of those systems. Sounds like the next step is to log a bug for fhr or a telemetry probe, and those teams will help you refine how to best collect that data.

I'm not sure there is a good equivalent on FxOS, you might be out of luck there.

On the server side, I believe we can observe if a link is actually clicked, I'm assuming that's this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1024946. If I understand correctly, in the short run all calls are initiated this way, but eventually they'll be initiated from accounts or msisdn so we should distinguish the clicked case on the server.
Flags: needinfo?(kparlante)
I'm pretty sure I know where to look for telemetry hooks etc, I'll take a look and spec it out tomorrow.
Assignee: nobody → standard8

Comment 9

5 years ago
Looking at the API more carefully, the server can track the number of "sharing" urls generated, and what user agent generated them. (The client needs to ask the server to create the url).
Telemetry seems the most appropriate solution given what we've done so far. I've just updated the user story with a rough outline of what I think is necessary.

The exact details on how counts work, may need to be confirmed with folks more familiar with telemetry when we implement it.
Assignee: standard8 → nobody
User Story: (updated)

Updated

5 years ago
Priority: P1 → P2
Target Milestone: mozilla33 → mozilla34
RT - Is this work covered elsewhere (like stats from TB) or do we need shared URLs from the client for Fx34?
Flags: needinfo?(rtestard)
Reporter

Comment 12

5 years ago
As far as I understand TB won't be able technically to see details on the shared URLs, this is really a client/server thing on our side.
Flags: needinfo?(rtestard)
Because this helps us gauge usage, I think this needs to be a P1.
Priority: P2 → P1

Comment 14

5 years ago
assuming doing via telemetry.
Whiteboard: p=? → [p=2]
Marco, can you add this to the current iteration?
Assignee: nobody → mdeboer
Status: NEW → ASSIGNED
Iteration: --- → 34.2
Flags: needinfo?(mmucci)
Whiteboard: [p=2] → [p=2] [qa-]
Forwarding to Gavin to consider including given your current assignments in the iteration.

(In reply to Mike de Boer [:mikedeboer] from comment #15)
> Marco, can you add this to the current iteration?
Iteration: 34.2 → ---
Points: --- → 2
Flags: needinfo?(mmucci) → needinfo?(gavin.sharp)
Un-assigning, because it's unlikely that I'll be working on this before my PTO starts.
Assignee: mdeboer → nobody
Status: ASSIGNED → NEW
Flags: needinfo?(gavin.sharp)
Flags: firefox-backlog+
Summary: Client needs to report shared URLs → Client needs to report number of shared URLs
Whiteboard: [p=2] [qa-] → [qa-]
Assignee: nobody → paolo.mozmail
Status: NEW → ASSIGNED
Iteration: --- → 34.3
Flags: qe-verify-
Whiteboard: [qa-]

Comment 18

5 years ago
When adding this piece of telemetry - if it's possible to also add how many URL's are created all up (in addition to how many are shared).  There was interest from TokBox to see how many created URLs are actually shared.
Assignee

Comment 19

5 years ago
(In reply to sescalante from comment #18)
> When adding this piece of telemetry - if it's possible to also add how many
> URL's are created all up (in addition to how many are shared).  There was
> interest from TokBox to see how many created URLs are actually shared.

This is my interpretation of the description in the User Story field. I haven't been able to look into the code in detail, but I believe this will be easy to implement.
Reporter

Comment 20

5 years ago
(In reply to :Paolo Amadini from comment #19)
> (In reply to sescalante from comment #18)
> > When adding this piece of telemetry - if it's possible to also add how many
> > URL's are created all up (in addition to how many are shared).  There was
> > interest from TokBox to see how many created URLs are actually shared.
> 
> This is my interpretation of the description in the User Story field. I
> haven't been able to look into the code in detail, but I believe this will
> be easy to implement.

I believe we have this already through LoopBasicMetrics.loop_urls_generated.json on the server?
(In reply to Romain Testard [:RT] from comment #20)
> (In reply to :Paolo Amadini from comment #19)
> > (In reply to sescalante from comment #18)
> > > When adding this piece of telemetry - if it's possible to also add how many
> > > URL's are created all up (in addition to how many are shared).  There was
> > > interest from TokBox to see how many created URLs are actually shared.
> > 
> > This is my interpretation of the description in the User Story field. I
> > haven't been able to look into the code in detail, but I believe this will
> > be easy to implement.
> 
> I believe we have this already through
> LoopBasicMetrics.loop_urls_generated.json on the server?

There's going to be a difference.

The server-side is going to give you the total urls generated for everyone.

The client side will give you the total for that client, and then how many are shared for that client. Hence you can get a ratio per client.


As not everyone will be submitting telemetry, you won't ever be able to get the total number of urls shared for everyone.
Reporter

Comment 22

5 years ago
Thanks.
Indeed it makes sense to also track via Telemetry the number of URLs created so we can correlate that info to the number of URLs shared (also using Telemetry).

The total number of URLs created collected on the Loop server side (LoopBasicMetrics.loop_urls_generated.json) will give us a reference to work out the ratio of client submitting through Telemetry when compared to overall clients so we can understand the real numbers of shared URLs better.

I created bug 1059186 to track implementation of logging of number of created URLs via Telemetry as it is different from this bug (logging of number of shared URLs via Telemetry).
Also note that we are tracking this number on the server-side (total number of created call-urls)

https://github.com/mozilla-services/loop-server/blob/master/loop/routes/call-url.js#L34
Assignee

Updated

5 years ago
Depends on: 1059754
Assignee

Comment 24

5 years ago
(In reply to Romain Testard [:RT] from comment #3)
> We need to report this per user agent (i.e Windows, Linux, Mac, FFOS, ...).

Just to clarify, Telemetry submissions do not currently distinguish between the three major desktop platforms, and there is no precedent for having an histogram per platform, especially because this wouldn't allow any in-depth analysis or correlation on the dashboards anyways.

My suggestion would be to only deal with a single measurement for Desktop in this bug. If more correlations are really required, then I believe Firefox Health Report provides what you want, though the FHR probes would need to go through a separate definition and approval process.
Assignee

Updated

5 years ago
Depends on: 1059186
Assignee

Comment 25

5 years ago
Clarified that this bug will only deal with reporting on Desktop.

The base patch will be on the newly filed bug 1059186.
Summary: Client needs to report number of shared URLs → Client needs to report number of shared URLs on Desktop
Assignee

Updated

5 years ago
No longer depends on: 1059754
Reporter

Comment 26

5 years ago
(In reply to :Paolo Amadini from comment #24)
> (In reply to Romain Testard [:RT] from comment #3)
> > We need to report this per user agent (i.e Windows, Linux, Mac, FFOS, ...).
> 
> Just to clarify, Telemetry submissions do not currently distinguish between
> the three major desktop platforms, and there is no precedent for having an
> histogram per platform, especially because this wouldn't allow any in-depth
> analysis or correlation on the dashboards anyways.
> 
> My suggestion would be to only deal with a single measurement for Desktop in
> this bug. If more correlations are really required, then I believe Firefox
> Health Report provides what you want, though the FHR probes would need to go
> through a separate definition and approval process.

OK agreed (we'll start with no distinction between desktop platforms) and thanks for clarifying.
A separate bug will deal with "Shared URL report" from FxOS devices.

I NI Maria for guidance on FxOS bug reference.
Flags: needinfo?(oteo)
Opened bug 1060386 to track the  number of shared URLs on FxOS
Thanks Romain!
Flags: needinfo?(oteo)
Note that the patch for Bug 1060610 adds a hook (handleLinkExfiltration) to panel.js, which is where this should be handled. I'm marking it as a dependency.
Depends on: 1060610
Iteration: 34.3 → 35.1
(In reply to :Paolo Amadini from comment #24)

> Just to clarify, Telemetry submissions do not currently distinguish between
> the three major desktop platforms, and there is no precedent for having an
> histogram per platform, especially because this wouldn't allow any in-depth
> analysis or correlation on the dashboards anyways.

Actually, that's not quite true. That was my understanding when I started bug 1023508, but I later noticed that the telemetry dashboard does let you drill down into per-platform data... Select a probe in telemetry.mozilla.org, change the "reason*" dropdown to "saved_session", and then additional dropdown are revealed that allow selecting from product, OS, and OS version. Surprise!

But you're right that Telemetry data can't be correlated beyond that, or even with other Telemetry probes.

[Bug 1023508 did per-platform probes, but I wouldn't do it that way again. And even so, we knew beforehand that what it was measuring would likely vary significantly between platforms, and it does indeed.]

Updated

5 years ago
Target Milestone: mozilla34 → mozilla35

Updated

5 years ago
Whiteboard: [loop-uplift]
Iteration: 35.1 → 35.2
Hi Paolo -- Are you planning to work on this during the 35.3 iteration, or should I find another owner?  (This is reporting info that Product wants as part of Loop MVP.)  Thanks.
Flags: needinfo?(paolo.mozmail)
Assignee

Comment 31

5 years ago
Posted patch The patch (obsolete) — Splinter Review
Was saving this for when the Contacts work has finished.
Attachment #8497404 - Flags: review?(MattN+bmo)
Flags: needinfo?(paolo.mozmail)
Assignee

Comment 32

5 years ago
To test this and bug 1059186 manually:
- Open "about:telemetry".
- Filter for the "LOOP_" histograms.
- Open the panel.
- Refresh the page and verify that the "1" column of "LOOP_CLIENT_CALL_URL_REQUESTS_SUCCESS" has increased.
- Copy or e-mail the call URL.
- Refresh the page and verify that the "1" column of "LOOP_CLIENT_CALL_URL_SHARED" has increased.
- Disconnect the network and open the panel again.
- Refresh the page and verify that the "0" column of "LOOP_CLIENT_CALL_URL_REQUESTS_SUCCESS" has increased.
Flags: qe-verify- → qe-verify+
Iteration: 35.2 → 35.3
Comment on attachment 8497404 [details] [diff] [review]
The patch

Review of attachment 8497404 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM. Thanks
Attachment #8497404 - Flags: review?(MattN+bmo) → review+

Updated

5 years ago
QA Contact: anthony.s.hughes
Assignee

Comment 36

5 years ago
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #35)
> Backed out for mochitest-bc orange.

Ah, it was a failure in the optimized builds only, this is why the try run didn't detect it. Probably there is some race condition to address in the telemetry tests, that doesn't occur with one histogram only.
Assignee

Comment 37

5 years ago
Posted patch Updated patchSplinter Review
It turns out the test was just wrong. New tryserver build:

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=d3ebc8924a85
Attachment #8497404 - Attachment is obsolete: true
Assignee

Updated

5 years ago
Keywords: checkin-needed
Assignee

Updated

5 years ago
Whiteboard: [loop-uplift] → [loop-uplift][checkin-needed to fx-team]
https://hg.mozilla.org/integration/fx-team/rev/9c56c35657b9
Keywords: checkin-needed
Whiteboard: [loop-uplift][checkin-needed to fx-team] → [loop-uplift]
https://hg.mozilla.org/mozilla-central/rev/9c56c35657b9
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
(In reply to :Paolo Amadini from comment #32)
> - Copy or e-mail the call URL.
> - Refresh the page and verify that the "1" column of
> "LOOP_CLIENT_CALL_URL_SHARED" has increased.
Clicking 'Copy' increases the variable. Clicking again on 'Copied!' button still increases the variable.
Thoughts ?
Flags: needinfo?(paolo.mozmail)
Same behavior if pressing CTRL-C multiple times on the same URL.
Assignee

Updated

5 years ago
Blocks: 1080661
Assignee

Comment 42

5 years ago
(In reply to Paul Silaghi, QA [:pauly] from comment #40)
> Clicking 'Copy' increases the variable. Clicking again on 'Copied!' button
> still increases the variable.
> Thoughts ?

That's a bug, thanks for finding it! Bug 1080661.
Flags: needinfo?(paolo.mozmail)
Besides that, everything seems to be ok. Tested on FF 35.0a1 (2014-10-09) Win 7, Ubuntu 13.04, OS X 10.10.
I'm marking this bug as verified fixed. The work continues in the follow up bug.
Status: RESOLVED → VERIFIED
Comment on attachment 8501030 [details] [diff] [review]
Updated patch

Approval Request Comment
Part of the staged Loop aurora second uplift set
Attachment #8501030 - Flags: approval-mozilla-aurora?
Comment on attachment 8501030 [details] [diff] [review]
Updated patch

mark earlier r+
Attachment #8501030 - Flags: review+
Flags: needinfo?(paul.silaghi)
Comment on attachment 8501030 [details] [diff] [review]
Updated patch

Approval previously granted to jesup to land this change as part of the second batch of Loop fixes to land on Aurora. Marking the approval after the fact.
Attachment #8501030 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Verified fixed FF 34b1 in bug 1080661
Flags: needinfo?(paul.silaghi)
Flags: in-moztrap-
You need to log in before you can comment on or make changes to this bug.