Write an airflow job to populate a derived database on hbase for TAAR

RESOLVED FIXED

Status

Data Platform and Tools
Add-on Recommender
P1
normal
RESOLVED FIXED
10 months ago
4 months ago

People

(Reporter: mlopatka, Assigned: Dexter)

Tracking

(Blocks: 1 bug)

Details

Attachments

(2 attachments)

(Reporter)

Description

10 months ago
Job to populate only the important columns for making addon recommendation into hbase instance that TAAR will use.

Should be modelled after this job: 
https://github.com/mozilla/telemetry-batch-view/blob/647abfb7f014995267360a50f8bacee788737b2f/src/main/scala/com/mozilla/telemetry/views/HBaseMainSummaryView.scala

documentation here: 
https://docs.google.com/document/d/1c6S786IA88Ga0NOlZiEEQj071AOTwyVZUeZYyihZNw0/edit
(Reporter)

Comment 1

10 months ago
depends on 1380713
Component: General → General
Product: Cloud Services → Data Platform and Tools
(Assignee)

Updated

10 months ago
Blocks: 1286215
Depends on: 1380713
(Assignee)

Comment 2

9 months ago
Other than the features outlined in bug 1380713, we also need to add the addonDetails column described here [1].

@aswan, as of now, the |addonDetails| field in telemetry seems to report information about installed legacy and disabled addons. Are you aware of any plan to not send this information in Firefox 57? Will we still have data about installed legacy addons when 57 goes live?

If you're not the right person to answer this, could you point me to who could?

[1] - http://gecko.readthedocs.io/en/latest/toolkit/components/telemetry/telemetry/data/main-ping.html#addondetails
[2] - https://dxr.mozilla.org/mozilla-central/search?q=setTelemetryDetails&case=true
Flags: needinfo?(aswan)

Comment 3

9 months ago
In 57, legacy addons will remain installed but disabled.  Among other things, this means that the author of a legacy extension can create a WebExtension version and provide it as an update, and the browser will install it just like any other extension update.  I don't know of any plans to change anything that would affect the information that is available to telemetry about disabled addons.
Flags: needinfo?(aswan)
(Assignee)

Updated

8 months ago
Depends on: 1390814
(Assignee)

Comment 4

8 months ago
Created attachment 8897941 [details] [review]
HBase Scalar Job PR
(Assignee)

Updated

8 months ago
Assignee: nobody → alessio.placitelli
Mentor: mlopatka
Priority: -- → P1
(Assignee)

Comment 5

8 months ago
Created attachment 8898867 [details] [review]
Airflow PR
(Assignee)

Updated

8 months ago
Attachment #8898867 - Flags: review?(fbertsch)
(Assignee)

Updated

8 months ago
Blocks: 1386274
(Assignee)

Updated

8 months ago
Attachment #8898867 - Flags: review?(fbertsch)
(Assignee)

Comment 6

8 months ago
The PRs were reviewed and merged and they are pushing data to the HBase instance.
Status: NEW → RESOLVED
Last Resolved: 8 months ago
Resolution: --- → FIXED
(Assignee)

Updated

8 months ago
Blocks: 1396549
(Assignee)

Updated

8 months ago
No longer blocks: 1396549
Moved to new component, per bug 1425844.
Component: General → Add-on Recommender
You need to log in before you can comment on or make changes to this bug.