1086706 - [Baloo] Access to Baloo aggregated data from bedrock

Status: RESOLVED WONTFIX

(Community Building :: Systems and Data, task)

Description

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Hi Pierros,

I know we've talked about this in previous meetings, and you suggested we pull data from Tableau. However, a Tableau REST API (see bug 1081284) will not be ready until early December, which will not work with our timeline. Please review the below request.

# Related Project

The primary project in question is Project MozID: Mozilla Identity Evolution (https://wiki.mozilla.org/Engagement/Campaigns/Mozilla_Identity_Evolution)

Prototype for reference: http://50.250.207.157/mozillaid/dev/prototypes/nick/clock/data.html

# Prototype Datasources

The current prototype is using an endpoint which is manually updated by pulling down text files from Tableau, and pasting into a Google Spreadsheet.

# Risks

The data will be used to generate the Mozilla logo, and possibly other properties on mozilla.org. These sites are highly-trafficked and demand reliable endpoints.

# Tableau Access

In bug 1081284 we reached out to Data and Metrics in an attempt to gain REST API access to Tableau. It was noted to us that the version of Tableau used would not be ready until Dec 1st. Which will not be compatible with our timeline.

# Request

We'd like a more reliable and automated service to update these datasets.

Specifically, access to the Baloo aggregated data used to represent these tabs via REST API, or some other method:
https://dataviz.mozilla.org/views/Active_Contributor_Dashboard/nums_total#1
https://dataviz.mozilla.org/views/Active_Contributor_Dashboard/nums_team#1
https://dataviz.mozilla.org/views/Active_Contributor_Dashboard/nums_source#1

We will only be accessing the data source - at most - once per day from bedrock.

https://wiki.mozilla.org/Baloo#Stack - this suggests that there is a Vertica data source that already houses aggregated data. Is it possible to interface with the data here?

--

Please let us know what the next steps are.

Thanks

Comment 1

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Hi Pierros, is this something that we just won't have time for in the next month? Or do we have to reach out to someone else? Thanks

Comment 2

[Pierros Papadeas [:pierros]]

Hey Cory, sorry for the slow response. MozFest drained all the time.

re: the request I am afraid we don't have the development resources to commit for timely delivery. The request is solid, and the functionality will be needed by other systems too, but we cannot deliver this within Q4.

I suggest you continue relying on manual updated data on doctodash.