1036050 - Improve .csv export for events data

Status: RESOLVED FIXED

(Webmaker Graveyard :: Events, defect)

Description

[Matt Thompson (@OpenMatt :OpenMatt)]

* This .csv data is important for key tracking and metrics. Improving the quality of the data coming out is important for the weekly reports Amira is expected to provide. 

Two issues:

1) "country" and "city" fields are often exporting as "null." These are key pieces of data we need. 


2) garbled formatting
* For example, the "number of attendees" field occasionally displays the event address instead
* Entire sections of the file break format and display the wrong data. Here's a screen grab: 
https://www.dropbox.com/s/mv70s1lt7n5wvn6/Screen%20Shot%202014-07-08%20at%204.24.19%20PM.png

Comment 1

[Matt Thompson (@OpenMatt :OpenMatt)]

* gvn: not sure who else to assign this to, so assigning to you for now -- please reassign as needed

Comment 2

[Gavin Suntop [:gvn]]

I'll see what I can do!

1. Unfortunately we can't guarantee these won't be null. It's up to the organizer to provide a location that country and city can be parsed from. Google's location API does the parsing. If a full address isn't provided it may not find a country or city (eg: "123 Fake Street" doesn't have a country or city).

2. I'm looking into this. I think I've got some ideas on improving it. Commas and line breaks in descriptions may be throwing things off.

Comment 3

[Gavin Suntop [:gvn]]

Attachment: https://github.com/mozilla/webmaker-events-service/pull/57

Comment 4

[Matt Thompson (@OpenMatt :OpenMatt)]

Thanks Gavin! I think (2) was the main pain-point, so progress on that will be awesome

Comment 5

[Gavin Suntop [:gvn]]

Attachment: https://github.com/mozilla/webmaker-events-service/pull/62

This should be event better...

Comment 6

[Gavin Suntop [:gvn]]

-t

Comment 7

[Chris DeCairos (:cade)]

Comment on attachment 8455561 [details] [review]
https://github.com/mozilla/webmaker-events-service/pull/62

This works, but it does kinda break down on array values like mentors and coorganizers. 

Is that going to be an issue? R+ if not.

Comment 8

[Matt Thompson (@OpenMatt :OpenMatt)]

+ Hannah

* Thanks Cade! 

* This is actually quite an important bug to get right -- many customers needs this CSV data in a better form than they're currently getting.

Comment 9

[Gavin Suntop [:gvn]]

I think the way it represents arrays is OK. It still shows their contents, just adds brackets around them too for some reason. I'm going to go ahead and merge and if anyone has problems with it we can revisit, but it should be a huge improvement.

Comment 10

[Brett Gaylor [:brett]]

Lets land and build, and see if this works for the party makers.

Comment 11

[Gavin Suntop [:gvn]]

Landed, not yet deployed.

Comment 12

[Gavin Suntop [:gvn]]

Attachment: https://github.com/mozilla/webmaker-events-service/pull/67

Actually you were right, Cade. The sub-objects for mentors/coorgs were rendering super badly. This should clean that up...

Comment 13

[Chris DeCairos (:cade)]

Comment on attachment 8456372 [details] [review]
https://github.com/mozilla/webmaker-events-service/pull/67

yes, this is soooo much better.

I did leave one suggestion in the pull request, but I leave it up to you whether or not it's worthwile, since it's just a cosmetic change to the code.

Comment 14

[Matt Thompson (@OpenMatt :OpenMatt)]

An-Me reports: 

'the data in the date column is "null"'


Thoughts?

Comment 15

[hannah:hkane]

Worked for me, though many of the date fields were null, especially those at the top of the list. When I scrolled down, I saw most of the date fields had valid data. I removed the line items with null in the date fields because, as I understood it, they're null for legacy reasons and are therefore old events anyway. 

I added that note to https://wiki.mozilla.org/Webmaker/Events_export

Comment 16

[Gavin Suntop [:gvn]]

That's correct, many legacy events won't have dates attached to them. Thanks for adding that to the wiki!

Comment 17

[Gavin Suntop [:gvn]]

Going ahead and re-closing this one. Feel free to re-open if there are other issues.