Currently, we store the breakpad revision that stackwalker uses in build-breakpad.sh, build it using a TaskCluster script, and then upload the build to a single public URL. Because of this, we cannot build a new revision of Breakpad in TaskCluster without making it the version we're using. Instead, we should be publishing Breakpad builds to a URL that includes the revision we're building from, so that we can pin stackwalker builds to a specific breakpad version and update it independently from Breakpad builds.  https://github.com/mozilla-services/socorro/blob/052d90bb2e94f581e61fae6c32c4aa523ad8d496/scripts/build-breakpad.sh#L17  https://github.com/mozilla-services/socorro/blob/052d90bb2e94f581e61fae6c32c4aa523ad8d496/scripts/breakpad-taskcluster.sh#L22  https://index.taskcluster.net/v1/task/project.socorro.breakpad.v1.builds.linux64.latest/artifacts/public/breakpad.tar.gz
While we're at it, we should update the set_up_stackwalk.sh script used in the Docker images to build Breakpad if the pinned version hasn't been uploaded. That will let us run CI against new pinned versions before we run the TC build.  https://github.com/mozilla-services/socorro/blob/master/docker/set_up_stackwalk.sh
This is something we should get to because it fixes some issues we have when updating breakpad. Making this a P2.
Priority: -- → P2
So I spent some time working on this, and it could work. But it'd be a little messy, and honestly I'm not sure if there's a whole lot of benefit from having TaskCluster build breakpad. Any solution that involves automation not downloading breakpad from URL requires that we build Breakpad ourselves, and if we can do that in our Docker setup, there's not much point in building breakpad separately from stackwalker anyway. Given that, I think the goal here should be to build Breakpad as part of our Docker-based infra. willkg: Thoughts?
I'm +1 on building Breakpad as part of our Docker-based infra. Maybe we should split out mdsw as a separate project with a separate "deploy" which would build it and put the build on S3. Then Socorro would have a file that specifies which version of mdsw/breakpad to use. That lets us test regressions easily and it doesn't mean we're building breakpad and mdsw during Socorro builds and tests which adds some time. I'm game for other options, too.
(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #4) > Maybe we should split out mdsw as a separate project with a separate > "deploy" which would build it and put the build on S3. Then Socorro would > have a file that specifies which version of mdsw/breakpad to use. That lets > us test regressions easily and it doesn't mean we're building breakpad and > mdsw during Socorro builds and tests which adds some time. I think this pairs nicely with long-term efforts to make mdsw callable via Python bindings instead of as a standalone executable, because we could distribute it as a compiled Python package and just pull it in via PyPI instead of special-casing it as we do currently. > I'm +1 on building Breakpad as part of our Docker-based infra. This seems like a tractable medium-term goal that we should do in the meantime.
I'm a little puzzled. I'm not suggesting we ever put what we've currently got on PyPI. I'm suggesting we move what we've got into a separate repository where on tagging it, it "deploys" in the sense that it builds a new binary and puts the binary on S3. I think the rewrite is a different bug. I don't have opinions on what that should look like, yet. I'm kind of loathe to put it on PyPI, though, unless it's generally useful to other groups.
This isn't something that *needs* to get done, so we're going to kick the can down the road a bit and make this a P3.
Priority: P2 → P3
Status: NEW → RESOLVED
Last Resolved: a month ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.