1587830 - Fx usage report's github repos dev container does not match dataproc's env

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, task, P3)

Description

[Harold Woo]

Background:
Repo https://github.com/mozilla/Fx_Usage_Report has a dev container used by tests. This currently doesn't match EMR or Dataproc but since the job has migrated to dataproc, we should match the dataproc env moving forward.

Dev container used by tests:
https://github.com/mozilla/Fx_Usage_Report/blob/master/Dockerfile

Dataproc uses debian9/ubuntu18

Dev container python reqs
https://github.com/mozilla/Fx_Usage_Report/blob/master/requirements.txt

should match:
Dataproc cluster init
https://github.com/mozilla/telemetry-airflow/blob/master/dataproc_bootstrap/fx_usage_init.sh

Plus script executed on cluster:
https://github.com/mozilla/telemetry-airflow/blob/master/jobs/fx_usage_report.sh#L7-L11

Comment 1

[Rob Miller [:rmiller]]

Hello,

The Mozilla Data Engineering organization is currently going through our extensive backlog, consisting of hundreds of issues stretching back for nearly 10 years. We've done a pass through all of the open bugzilla bugs and have identified and tagged the ones that we think are relevant enough to still need attention. The rest, including the bug with which this comment is associated, we are closing as "WONTFIX" in a single bulk operation.

If you feel we have closed this (or any) issue in error, please feel free to take the following actions:

• Reopen the bug.
• Edit the bug to add the string [dataplatform] (including the brackets) to the Whiteboard field. (Note that you must edit the Whiteboard, not the similarly named QA Whiteboard.)

Doing this will ensure that we see the bug in our weekly triage process, where we will decide how to proceed.

Thank you.