1542394 - rewrite update_signatures crontabber job as django command

Status: RESOLVED FIXED

(Socorro :: General, task, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

"update_signatures" is a crontabber job. The code is here:

https://github.com/mozilla-services/socorro/blob/master/socorro/cron/jobs/update_signatures.py

This bug covers rewriting that as a Django command with tests run by the Django cronrun command.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this a P2 to do soon.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4887: bug 1542394: rewrite updatesignatures as Django command

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #4887: "bug 1542394: rewrite updatesignatures as Django command" in 3b46bc8.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Landed this and it ran on stage and kicked this up:

https://sentry.prod.mozaws.net/operations/socorro-new-stage/issues/5519928/

ValueError: Unable to parse connection string: ":11211"
  File "crashstats/cron/management/commands/cronrun.py", line 152, in _run_one
    self._run_job(job_spec, *cmd_args, **cmd_kwargs)
  File "crashstats/cron/management/commands/cronrun.py", line 196, in _run_job
    return call_command(job_spec['cmd'], *cmd_args, stdout=self.stdout, **cmd_kwargs)
  File "django/core/management/__init__.py", line 148, in call_command
    return command.execute(*args, **defaults)
  File "django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "crashstats/crashstats/management/commands/updatesignatures.py", line 113, in handle
    resp = api.get(**params)
  File "crashstats/supersearch/models.py", line 178, in get
    return super().get(**kwargs)
  File "crashstats/crashstats/models.py", line 482, in get
    return self._get(expect_json=expect_json, **kwargs)
  File "crashstats/crashstats/models.py", line 544, in _get
    expect_json=expect_json,
  File "crashstats/crashstats/models.py", line 352, in inner
    result = method(*args, **kwargs)
  File "crashstats/crashstats/models.py", line 428, in fetch
    result = cache.get(cache_key)
  File "django/core/cache/backends/memcached.py", line 71, in get
    val = self._cache.get(key)
  File "django/core/cache/backends/memcached.py", line 162, in _cache
    self._client = self._lib.Client(self._servers, **client_kwargs)
  File "site-packages/memcache.py", line 215, in __init__
    self.set_servers(servers)
  File "site-packages/memcache.py", line 294, in set_servers
    for s in servers]
  File "site-packages/memcache.py", line 294, in <listcomp>
    for s in servers]
  File "site-packages/memcache.py", line 1347, in __init__
    raise ValueError('Unable to parse connection string: "%s"' % host)

I'm not sure why that happened. I'll have to look into it. Maybe caching isn't set up right in when running commands somehow? Maybe caching environment variables aren't available in the crontabber container?

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

Regarding comment #4, configuration for caching wasn't set in the crontabber container. That got worked out in the ops repos.

Pushed this to prod just now.