Closed Bug 1410254 Opened 7 years ago Closed 7 years ago

retention_v1: `app_version` only contains two values

Categories

(Data Platform and Tools :: General, enhancement, P1)

Product:
Data Platform and Tools
Component:
General
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: amiyaguchi, Assigned: amiyaguchi)

References

Details

Attachments

(1 file)

Link to GitHub pull-request: https://github.com/mozilla/python_mozetl/pull/158
49 bytes, text/x-github-pull-request
Details | Review 
`app_version` only contains the values ("app_version", "older"). This is caused by different behavior between spark 2.2 and spark 2.0.2 in `pyspark.sql.functions.when`.

The value should be wrapped in the `col` operator, and the EMR release should use EMR 5.8.0.
> SELECT distinct(app_version) from retention
Assignee: nobody → amiyaguchi
Blocks: 1381840
Status: NEW → ASSIGNED
Priority: -- → P1
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Component: Datasets: General → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: