Closed Bug 641640 Opened 14 years ago Closed 12 years ago

Ensure VAMO is set up to handle Personas pings

Categories

(addons.mozilla.org Graveyard :: Code Quality, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED
2013-04-04

People

(Reporter: clouserw, Assigned: cvan)

References

Details

Right now when you install a persona, the JSON blob that defines the persona also sends an update_check URL. That is http://www.getpersonas.com/update_check right now. Since GP is scheduled to be put down, we need to send these update pings somewhere else, the natural spot would be versioncheck.amo. This bug is to ensure that we're all onboard with that plan and that you can handle the GP ping format to that location.
We need to study this. The biggest problem is that vamo is by far the largest data source that we parse, and the current ETL that is responsible for parsing personas might not be able to handle that volume. Especially since we are using a different ETL to process the other data in vamo logs. I'll work with Nelson this week to sort something out on it. What is the ETA for EOL?
Sometime in Q3, but I'd like to get our plan ready well before that.
Okay, we can definitely work something out given a few weeks of breathing room after Fx4. :)
Any news?
(In reply to comment #4) > Any news?
We can point personas and lw themes log processing to vamo files, I'll look into that next monday. My only concern is with filesizes, VAMO logs are huge and our current etl's performance may get hurt.
Well then I suggest you take the time to integrate the processing into the existing ETL that processes vamo.
We currently parse requests from getpersonas.com and getpersonas-ssl.com with requests like GET\s/([a-zA-Z\-]*)/update_check/(\d*), where the first field is the locale and the second is the persona id. This ping occurs once per day when Firefox checks whether there's an updated version of that specific persona, and this is the ping we use to determine persona usage. What would the new ping look like? As long as we have a pattern to look for on the vamo logs we can integrate this ping on vamo parsing and phase out our current personas and lightweight themes processing.
The plan was just to change the location. So, I'd expect: versioncheck.addons.mozilla.org/:lang/update_check/:id It would be nice to say something about personas in there though. I'll see what I can do.
Hi Rob. This is low priority for 6.4.0 but is potentially a quick bug and a good break from other stuff. The bug is essentially taking all the logic in this short file [1] and putting it in [2] which you've got experience with already. We just want VAMO to be able to accept persona pings and respond intelligently. If you throw /personas/ after /:lang/ in the URL in comment 10 I'd be happy. :) Once this ^^ is done we can resolve this bug and open a new one for metrics to make sure they do whatever process they need on their end. [1] http://viewvc.svn.mozilla.org/vc/projects/getpersonas.com/trunk/server/update_check.php?view=markup [2] https://github.com/mozilla/zamboni/blob/master/services/update.py
Assignee: deinspanjer → chudson
Priority: P1 → P5
Whiteboard: [file metrics bug when resolved]
Target Milestone: Q3 2011 → 6.4.0
Target Milestone: 6.4.0 → 6.4.1
work in progress: https://github.com/robhudson/zamboni/commit/44fe6902 A couple outstanding questions or issues here... 1) Do we need to continue to escape slashes like the current JSON output does, or is this a PHP artifact? The patch as it is doesn't do this. 2) The patch needs to set a few settings to the persona CDN and site URL. 3) The data URL requires the persona files exist on the same server to calculate the base64 string, so this is flagged as "TODO". 4) There are some migration questions that may affect this code.
When you land this can you let us know the URLs requests will be coming in on? Also we're going to need to be able to 301 stuff from getpersonas.com to here so the URL will need to support that. We can make that a separate bug if this one is ready to land and can't handle that though.
Answer to #1: No, we don't need to escape slashes. I got a quick review from Andy on Friday and would like to do the following before I push this in: 1. Use more things from services/utils.py that is shared amongst the scripts there. 2. Write tests. Bumping to 6.4.2
Target Milestone: 6.4.1 → 6.4.2
Target Milestone: 6.4.2 → 6.4.3
Target Milestone: 6.4.3 → 6.4.4
Target Milestone: 6.4.4 → 6.4.5
Target Milestone: 6.4.5 → 6.4.7
Target Milestone: 6.4.7 → 6.4.8
Target Milestone: 6.4.8 → 6.5.0
Target Milestone: 6.5.0 → 6.4.9
Summary: Ensure VAMO is setup to handle Personas pings → Ensure VAMO is set up to handle Personas pings
Target Milestone: 6.4.9 → ---
bug 847193 gives us a=ops for moving this traffic to VAMO
Priority: P5 → P3
Whiteboard: [file metrics bug when resolved]
Target Milestone: --- → 2013-03-14
Rebased onto current master and incorporated changes based on Andy's feedback: https://github.com/robhudson/zamboni/commit/149116baafe25ead7a45e7e5f04c7bcc909eebd4 This depends on a few bugs for certain files to be in place: * bug 846088 * bug 846957
TODO: * Tests * Use simpler path structure with ID.
Priority: P3 → P1
Target Milestone: 2013-03-14 → 2013-03-21
Assignee: robhudson.mozbugs → cvan
Status: NEW → ASSIGNED
Target Milestone: 2013-03-21 → 2013-03-28
Target Milestone: 2013-03-28 → 2013-04-04
https://github.com/mozilla/zamboni/commit/73bea26 Thanks, Rob, for doing most of the work here - much appreciated! I shall now work with IT to stand up this wsgi application.
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.