1410976 - Add Breakpad changeset to TaskCluster build URL

Status: RESOLVED WONTFIX

(Socorro :: General, task, P3)

Description

[Osmose [:osmose, :mkelly]]

Currently, we store the breakpad revision that stackwalker uses in build-breakpad.sh[1], build it using a TaskCluster script[2], and then upload the build to a single public URL[3]. Because of this, we cannot build a new revision of Breakpad in TaskCluster without making it the version we're using. 

Instead, we should be publishing Breakpad builds to a URL that includes the revision we're building from, so that we can pin stackwalker builds to a specific breakpad version and update it independently from Breakpad builds.


[1] https://github.com/mozilla-services/socorro/blob/052d90bb2e94f581e61fae6c32c4aa523ad8d496/scripts/build-breakpad.sh#L17

[2] https://github.com/mozilla-services/socorro/blob/052d90bb2e94f581e61fae6c32c4aa523ad8d496/scripts/breakpad-taskcluster.sh#L22

[3] https://index.taskcluster.net/v1/task/project.socorro.breakpad.v1.builds.linux64.latest/artifacts/public/breakpad.tar.gz

Comment 1

[Osmose [:osmose, :mkelly]]

While we're at it, we should update the set_up_stackwalk.sh[1] script used in the Docker images to build Breakpad if the pinned version hasn't been uploaded. That will let us run CI against new pinned versions before we run the TC build.

[1] https://github.com/mozilla-services/socorro/blob/master/docker/set_up_stackwalk.sh

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

This is something we should get to because it fixes some issues we have when updating breakpad. Making this a P2.

Comment 3

[Osmose [:osmose, :mkelly]]

So I spent some time working on this, and it could work. But it'd be a little messy, and honestly I'm not sure if there's a whole lot of benefit from having TaskCluster build breakpad. Any solution that involves automation not downloading breakpad from URL requires that we build Breakpad ourselves, and if we can do that in our Docker setup, there's not much point in building breakpad separately from stackwalker anyway.

Given that, I think the goal here should be to build Breakpad as part of our Docker-based infra. 

willkg: Thoughts?

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm +1 on building Breakpad as part of our Docker-based infra.

Maybe we should split out mdsw as a separate project with a separate "deploy" which would build it and put the build on S3. Then Socorro would have a file that specifies which version of mdsw/breakpad to use. That lets us test regressions easily and it doesn't mean we're building breakpad and mdsw during Socorro builds and tests which adds some time.

I'm game for other options, too.

Comment 5

[Osmose [:osmose, :mkelly]]

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #4)
> Maybe we should split out mdsw as a separate project with a separate
> "deploy" which would build it and put the build on S3. Then Socorro would
> have a file that specifies which version of mdsw/breakpad to use. That lets
> us test regressions easily and it doesn't mean we're building breakpad and
> mdsw during Socorro builds and tests which adds some time.

I think this pairs nicely with long-term efforts to make mdsw callable via Python bindings instead of as a standalone executable, because we could distribute it as a compiled Python package and just pull it in via PyPI instead of special-casing it as we do currently.

> I'm +1 on building Breakpad as part of our Docker-based infra.

This seems like a tractable medium-term goal that we should do in the meantime.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm a little puzzled. I'm not suggesting we ever put what we've currently got on PyPI. I'm suggesting we move what we've got into a separate repository where on tagging it, it "deploys" in the sense that it builds a new binary and puts the binary on S3.

I think the rewrite is a different bug. I don't have opinions on what that should look like, yet. I'm kind of loathe to put it on PyPI, though, unless it's generally useful to other groups.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

This isn't something that *needs* to get done, so we're going to kick the can down the road a bit and make this a P3.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

We don't use Taskcluster anymore for building the breakpad client, so I'm going to WONTFIX this.