1316185 - fold socorrolib back into socorro

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

At the beginning of this year, we decided to split Socorro up into multiple components such that they could have their own deploy cycles and it'd make it easier to set up an infrastructure that was AWS friendly. The thinking was that we'd extract a common library that all the components use into a separate repository and then extract components one at a time and have them depend on the common library.

This work was successful in the sense that it begat socorrolib. However, the first component we worked on extracting was the collector and instead of just extracting it and using socorrolib, we did a rewrite. Further, it's looking increasingly likely that we're going to rebuild other components from or nearly from scratch.

Because of that, we're not getting much value out of having an extracted socorrolib. Further, we periodically encounter pain points related to it being separate, the most recent being bug #1315308.

This bug covers figuring out whether it's worth folding socorrolib back in and if so, doing that work.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

I think the work is roughly like this:

1. copy the code and tests back into socorro repo
2. update all the import statements to point to the correct Python modules
3. push all that out to -stage and -prod
4. update the configuration on -stage and -prod to point to the new locations of things
5. remove the socorrolib requirement
6. deprecate the socorrolib repo

I'm +1 on folding it back in sooner rather than later because it's not helping us now, it doesn't help us with our future plans, merging it in means one less thing to deal with and it might make it easier to extract the processor.

I'm game for doing the work if that helps.

lonnen, peterbe, adrian: What do you think?

Comment 2

[Lonnen :lonnen]

Now that we're committed to rewriting, socorrolib isn't useful divorced from the rest of the application.

Having it independent is not making it harder to extract the processor. It may still be worthwhile to merge it in to save some overhead for the next few months on edits impacting the processor and collector in the meanwhile.

If this is a < 1 week project, it is probably worthwhile.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

I don't think it'd take a solid week of work to do, but it might take a week from beginning to end because some of the steps take a day or so to work through. I can do a back-of-the-envelope estimate tomorrow.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

We'd move:

* socorrolib/lib/ -> socorro/lib/
* socorrolib/app/ -> socorro/app/
* socorrolib/unittest/ -> socorro/unittest/

I see 300-400 instances of "socorrolib" in the socorro repo. Seems straight-forward search-and-replace.

It shows up once in configuration:

socorro/submitter/producer_consumer.producer_consumer_class     socorrolib.lib.threaded_task_manager.ThreadedTaskManager

Pretty sure that's used by the stage submitter which doesn't get updated during deploys.

Given that, we could probably do it this way:

1. copy the code and tests back into socorro repo
2. update anything pointing to socorrolib to point to the new paths (import statements, configuration defaults, documentation, etc)
3. remove socorrolib requirement
4. PR review, fixes from comments, etc
5. push all that to -stage
6. system test on -stage
7. push to -prod
8. deprecate socorrolib repo
9. update the stage submitter code and configuration

My back-of-the-envelope is that it'd take a day or so to do the code changes and put up a PR. Then it follows the regular progression of things.

Comment 5

[Peter Bengtsson [:peterbe]]

Go for it!

Comment 6

[[DEACTIVATED] Adrian Gaudebert]

Yep, +1!

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this to work on now.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Two more things:

1. need to check socorro-infra
2. need to check any .ini files that might be floating around that aren't in socorro or socorro-infra repositories

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: Link to Github pull-request: https://github.com/mozilla/socorro/pull/3584

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

I checked the socorro-infra repository and didn't see any instances of "socorrolib", so I think there's nothing to do there.

One other thing to do--after we land the PR and test no -stage, we should send an email to socorro-dev about the change and telling people to update their local instances. Probably the only thing they would need to update is instances of "socorrolib" in configuration in their dev environments.

Comment 11

[Lonnen :lonnen]

summarizing irc --

The patch is r+, there are no external services that use the python path directly. I had concerns around migrations because of https://bugzilla.mozilla.org/show_bug.cgi?id=1300367#c10, but that issue was not caused by socorrolib. We are now waiting for stage to clear.

Comment 12

[Will Kahn-Greene [:willkg] ET needinfo? me]

We need to update stage submitter (super sekret submitter node) and crontabber (admin node). Those don't get updated during deploy, so I'll do them by hand afterwards.

Comment 13

[Will Kahn-Greene [:willkg] ET needinfo? me]

Test plan! https://public.etherpad-mozilla.org/p/socorro-socorrolib-testplan-2016

Comment 14

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/9c3ba26e96b42836edbaf5f6d5fd53d41aca7201
Fixes bug 1316185: fold socorrolib back in (#3584)

At the beginning of the year, we extracted socorrolib from socorro with
the thinking that we'd use socorrolib as a requirement for future
projects. Later, we decided to do those projects in other ways, so
then the value proposition of maintaining socorrolib as a separate
repository wasn't enthusing.

This merges that code back in. It was done in the following steps:

1. copy the relevant directories from socorrolib/ to socorro/
2. do a global search-and-replace of "socorrolib" to "socorro"

It's pretty straight-forward--nothing fancy.

Comment 15

[Will Kahn-Greene [:willkg] ET needinfo? me]

Reopening this because I still need to:

1. update stage submitter and configuration
2. update crontabber

Comment 16

[Will Kahn-Greene [:willkg] ET needinfo? me]

I merged the code and Peter updated the -stage admin node and crontabber.

We did the system test and everything looked fine.

We should wait until tomorrow to see if there are any crontabber issues.

If everything looks good, I'll push to -prod.

Pushing to -prod involves the following:

1. pushing to -prod
2. update -prod admin node and crontabber
3. updating the consul configuration (need to write out what that entails--I'll do that in another comment)
4. update the -stage submitter

Comment 17

[Will Kahn-Greene [:willkg] ET needinfo? me]

Just now:

1. I pushed it to -prod and watched datadog graphs--seems fine.
2. Peter updated the -prod admin node and crontabber.

The consul change is this:

consulate kv set socorro/submitter/producer_consumer.producer_consumer_class "socorro.lib.threaded_task_manager.ThreadedTaskManager"

After doing that, I'll update the -stage submitter.

Comment 18

[Will Kahn-Greene [:willkg] ET needinfo? me]

I updated the stage submitter. Curiously the RPM is socorro-252 and not socorro-253. 252 is the previous tag and 253 is the current one. I verified that the code in /data/socorro didn't have socorrolib nor did it install socorrolib into the virtualenv. Curious.

Anyhow, that update worked in the update sense, but stage submitter wasn't working because we didn't have Python 2.7.11 installed on this box so the virtualenv was pointing to non-existing thing. The previous version of socorro that was on this box was 191 which predates the Python 2.7.11 upgrade.

I thought about copying Python 2.7.11 from somewhere else, but then got lazy and installed the packages I needed to build and install it on this box:

yum install bzip2-devel epel-release gcc-c++ git libffi-devel libxml2-devel libxslt-devel make mercurial openssl-devel python-devel sqlite-devel

Then I downloaded the tarball, built Python 2.7.11 and altinstalled it.

That worked in the Python sense, but submission was dying with certificate errors:

CertificateError: hostname 'crash-reports.mocotoolsstaging.net' doesn't match either of 'crash-stats.allizom.org', 'crash-reports.allizom.org'

Then I changed the configuration:

consulate kv set socorro/submitter/destination.url "https://crash-reports.allizom.org/submit"

and said the magic words and crossed my fingers, but in my heart I waited for the next problem.

... but there wasn't one. Everything worked. That fixed it.

That was the last bit for this bug. The show is over. Everything is done. It's time to go home. Please exit the theater using the doors to the left and right side of the stage. Don't forget your belongings. Place your trash in the appropriate receptacles.

Good night.