1150237 - deploy infernyx 0.1.x to stage and then prod

Status: RESOLVED FIXED

(Content Services Graveyard :: Tiles: Ops, defect)

Description

[Daniel Thorn [:relud]]

infernyx 0.1.39 adds a #! to csdash.py, among other things.

Comment 1

[Daniel Thorn [:relud]]

deployed to stage, but unverified there. i'd like to force parsing at least one impression and one stats log, to make sure it can, before continuing to prod. This is because last time it was deployed to prod, everything looked fine in stage, but logs didn't actually get parsed.

Comment 2

[Daniel Thorn [:relud]]

i've manually added the #! in prod, so that the csdash.py script will work until this can be deployed.

Comment 3

[Daniel Thorn [:relud]]

I forced impression_stats to run by reducing min_blobs to 1 and executing:

curl -i -X POST -d '{"view": 1, "locale": "en-US", "tiles": [{}]}' https://onyx_tiles.stage.mozaws.net/v2/links/view

to induce impression_stats blobs. impression_stats runs automatically in stage, but it throws this error (with context):

015-04-03 17:33:29,227 INFO  4013 [root] impression_stats@594:141e5:5c152: Processed 1 records in 3 keysets.
2015-04-03 17:33:29,228 INFO  4013 [root] impression_stats@594:141e5:5c152: Cleaning up tmp files: /tmp/newtab_statsrwL5sR (leaving s3: ('tiles-resources-stage-infernyxs3-w7yqgazowzp0', u'impression_stats@594:141e5:5c152-/tmp/newtab_statsrwL5sR'))
2015-04-03 17:33:29,228 INFO  4013 [inferno.lib.job] Worker: impression_stats@594:141e5:5c152 stage= job.purge 
2015-04-03 17:33:29,386 INFO  4013 [inferno.lib.job] Worker: impression_stats@594:141e5:5c152 stage= job.archive 
2015-04-03 17:33:29,546 INFO  4013 [inferno.lib.job] Worker: impression_stats@594:141e5:5c152 stage= job.cleanup 
2015-04-03 17:33:29,703 ERROR 4013 [rules] Error writing stats Unable to access resource (disco://discoelb/disco/ctrl/jobinfo?name=impression_stats@594:141e5:5c152): Not found. (404)
2015-04-03 17:33:29,703 INFO  4013 [inferno.lib.job] Worker: impression_stats@594:141e5:5c152 stage= job.done 
2015-04-03 17:33:29,703 INFO  4013 [inferno.lib.job] Finished job impression_stats@594:141e5:5c152
2015-04-03 17:38:24,064 INFO  10624 [inferno.lib.daemon] trying job impression_stats

I think that's fine, but I'm not sure, :tspurway could you verify that this looks good?

Comment 4

[Daniel Thorn [:relud]]

changing title to deploy 0.1.x because if we push 0.1.39 to prod it would overwrite impression_stats max_blobs from 1500 down to 300, and that might not be desirable. So potentially this should wait until https://bugzilla.mozilla.org/show_bug.cgi?id=1150950 is resolved and a new version is released before going to prod.

Comment 5

[Tim Spurway [:tspurway]]

This looks good, :relud.

Comment 6

[Daniel Thorn [:relud]]

it is done

Comment 7

[Daniel Thorn [:relud]]

whoops, wrong bug