1515805 - update breakpad client to b988fa74ec18de6214b18f723e48331d9a7802ae

Status: RESOLVED FIXED

(Socorro :: Processor, task, P1)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Per bug #1513284, minidump-stackwalker in Socorro is using an old breakpad client and it needs to be updated.

https://bugzilla.mozilla.org/show_bug.cgi?id=1513284#c15

> (In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #13)
> > Socorro is using breakpad rev a61afe7a3e865f1da7ff7185184fe23977c2adca.
> > Maybe we need to update it?
> 
> Yeah, it's pretty old and it's missing the changes to understand
> AArch64 minidumps using Microsoft format. Our checkout in 
> mozilla-central is at version 1459e5df74dd03b7d3d473e6d271413d7aa98a88
> plus some of our patches. That should be enough and as Mike mentioned
> the one in tooltool should also be good enough.

This bug covers updating the breakpad client.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Hrm.... The depot_tools don't work in Python 3, so we can't build the breakpad client in the app Docker image we've got. Seems like if we have a breakpad client already, then building stackwalker works--that's how deploys have been working so far.

I think Ted said when he's debugging breakpad/stackwalker issues, he builds it locally. That works for me, too. However, if someone pops in for whom that doesn't work, then we might want to set up a "breakpad/stackwalk-debug" image.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Rrr... I feel silly--I can't build stackwalker outside the container and then run it inside the container because I've got different versions of the libs.

I'm going to spend some time tweaking the scripts so we can build and run everything inside the container.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4754: bug 1515805: fix breakpad client building in image; add scripts

Comment 4

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/7ee852e3d530cc95a2bdab91a6c00187a6bdf882
bug 1515805: fix breakpad client building in image; add scripts

Building the breakpad client requires using depot_tools which only work with
Python 2. This fixes the build-breakpad.sh script so it makes sure depot_tools
are using Python 2 in the Docker container.

This adds a scripts/run_mdsw.sh script which runs minidump-stackwalker just
like we do in the processor, but outside of the processor. This makes it
a lot easier to debug issues with minidump-stackwalker.

https://github.com/mozilla-services/socorro/commit/b970b6c97c908e76a990f94561d7d62bdd893408
Merge pull request #4754 from willkg/1515805-breakpad

bug 1515805: fix breakpad client building in image; add scripts

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I've got this version of breakpad client working with minidump-stackwalker. I have taskcluster permissions to run the taskcluster task.

Next steps:

1. run the taskcluster task which will update the binary to 1459e5d
2. land the changes in the master branch
3. that'll trigger a stage deploy with the new binary
4. test everything out on stage
5. do a prod deploy

I think I can do all that on Monday.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4763: fix bug 1515805: update breakpad client to 1459e5df

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4765: bug 1515805: update github user in url

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

PR 4763 landed in https://github.com/mozilla-services/socorro/commit/adf73fa1bb6ccd9c18be605ff2dcea8269dcd2f6

PR 4765 landed in https://github.com/mozilla-services/socorro/commit/7d028cb3f683aa7a75298981c83b179fc5cf9900

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

Bug #1518947 covers updating the client in firefox to b988fa74ec18de6214b18f723e48331d9a7802ae. We could switch this bug to do that or we could do that in a later bug. Figured I'd mention it in a comment somewhere so I don't forget.

Comment 10

[Osmose [:osmose, :mkelly]]

I still have permissions to run this job, and attempted to, and am getting errors that look like this:

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://github.com/google/protobuf.git /home/worker/breakpad/src/src/third_party/protobuf/_gclien
t_protobuf_TM1TQT' in '/home/worker/breakpad'
[0:00:00] Cloning into '/home/worker/breakpad/src/src/third_party/protobuf/_gclient_protobuf_TM1TQT'...
[0:00:00] fatal: unable to access 'https://github.com/google/protobuf.git/': SSL connect error

It appears that the curl/libcurl (which git uses) being used in the gecko-misc image is so ancient as to not support TLS v1.2. So far I've had no luck trying to update it, and instead may try using a different image.

Comment 11

[Osmose [:osmose, :mkelly]]

Attachment: pr 4766: Bug 1515805: Build Breakpad using Socorro Docker image in TaskCluster.

Comment 12

[Osmose [:osmose, :mkelly]]

I've successfully updated the Breakpad version to 1459e5df74dd03b7d3d473e6d271413d7aa98a88 now that we resolved the build issues.

RE: Comment #9, we might as well use this bug rather than bother making another one, so all that's left is to land the hash change and to run the TaskCluster job one more time. We'll need some other change to land to get the new binary on stage before we can test it out.

Comment 13

[Osmose [:osmose, :mkelly]]

Attachment: pr 4768: Fix bug 1515805: Update Breakpad version.

Comment 14

[Osmose [:osmose, :mkelly]]

The breakpad version has been updated, and this is deploying to stage as we speak.