1387316 - [Form Autofill] Create dashboard for the form autofill type counts

Status: NEW

(Toolkit :: Form Autofill, enhancement, P3)

Description

[Steve Chung [:steveck]]

Follow up for Bug #990200 that we'll need the generate a dashboard based on the collected data. We already have dashboards in telemetry.mozilla.org[1], but the finalized version will still need PM's input and set new queries in s.t.m.o.

[1] https://goo.gl/PNQRB1 :The manual input count, you can switch to other type like autofill or autofill-update

Comment 1

[chsiang]

Attachment: IMG_6879.JPG

Comment 2

[Matthew N. [:MattN]]

Attachment: ATMO notebook WIP

Comment 3

[Steve Chung [:steveck]]

Based on the discussion with Cindy, we'll schedule a daily ATMO report for her.

Comment 4

[Steve Chung [:steveck]]

Attachment: Form_Autofill.ipynb

Hi MattN, I revise your WIP by adding the form filling time telemetry and dynamic submission date(I set it to 2 days before yesterday). Feel free to give any suggestion if you have time, thanks.

Comment 5

[Steve Chung [:steveck]]

Here is the result[1] I scheduled for daily dashboard, but I'm not sure if it's a permalink and refreash everyday...

[1] https://s3-us-west-2.amazonaws.com/telemetry-public-analysis-2/formautofill-telemetry-dashboard/data/Form_Autofill.ipynb

Comment 6

[Steve Chung [:steveck]]

Here is the result[1] I scheduled for daily dashboard in atmo service, but I'm not sure if it's a permalink and refreash everyday...

[1] https://nbviewer.jupyter.org/url/s3-us-west-2.amazonaws.com/telemetry-public-analysis-2/formautofill-telemetry-dashboard/data/Form_Autofill.ipynb

Comment 7

[Matthew N. [:MattN]]

Comment on attachment 8929402 [details]
Form_Autofill.ipynb

Hi Steve, nice job adding the filling time graphs.

>    "target_date = datetime.datetime.now() - datetime.timedelta(days=3)\n",
>    "target_date = target_date.strftime(\"%Y%m%d\")\n",
>    "print(\"The submission date is \" + target_date)\n",
>    "\n",
>    "pings = Dataset.from_source(\"telemetry\") \\\n",
>    "    .where(docType=\"main\") \\\n",
>    "    .where(submissionDate=target_date) \\\n",

I would recommend giving a lambda here so a range of days can be considered since weekend users are often different than weekday users. To address that I would recommend 7 or 14 days.

Example: .where(submissionDate=lambda x: x >= target_date_begin and x <= target_date_end)

>    "    .where(appUpdateChannel=\"beta\") \\\n",

This should be "release" probably

>    "    .records(sc, sample=0.1)"

You will want to lower this for release or it will take too long. Try 0.01 to start

>    "df_manual = pd.DataFrame(fill_type_manual_time_data, columns=['manual'])\n",
>    "df_manual.plot.bar(title=\"Manual filling time(ms) and count\", figsize=[15, 7])"

I would recommend using iplot so that the graphs are interactive otherwise I find it hard to read values from narrow bar graphs.

I didn't look closely at the rest since I don't do this stuff enough to be able to notice subtle issues via code review.

(In reply to Steve Chung [:steveck] from comment #6)
> I'm not sure if it's a permalink and refreash everyday...

Yes, it is.

Comment 8

[Matthew N. [:MattN]]

Steve, can you please update the dashboard with the above changes? The `beta` data isn't super useful as-is.

Comment 9

[Steve Chung [:steveck]]

(In reply to Matthew N. [:MattN] (PM if requests are blocking you) from comment #8)
> Steve, can you please update the dashboard with the above changes? The
> `beta` data isn't super useful as-is.

I scheduled another dashboard for release here[1] with 0.01 sample but it has some error while rendering the form filling time chart. Will try to fix it soon.

[1] https://nbviewer.jupyter.org/url/s3-us-west-2.amazonaws.com/telemetry-public-analysis-2/formautofill-telemetry-address-release-dashboard/data/Form_Autofill.ipynb