Closed Bug 1451675 Opened 7 years ago Closed 6 years ago

Test telemetry-streaming with Spark v2.3

Categories

(Data Platform and Tools Graveyard :: Datasets: Error Aggregates, enhancement, P2)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: akomar, Assigned: akomar)

Details

We'd like to be sure that telemetry-streaming jobs run correctly on Spark 2.3 before it's upgraded. Current status: * build and tests pass * ErrorAggregator works on Databricks Worth checking: * EventsToAmplitude on EMR * ExperimentsErrorAggregator on EMR
Until we're done with this, we can keep master built with v2.2 and still deploy ErrorAggregator to v2.3 runtime - it doesn't seem to be a problem in this case.
Priority: -- → P2
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
EventsToAmplitude and ExperimentsErrorAggregator are bumped to Spar 2.3 on Airflow.
Product: Data Platform and Tools → Data Platform and Tools Graveyard
You need to log in before you can comment on or make changes to this bug.