1675263 - Unbreak the system-symbols-* tasks

Status: RESOLVED FIXED

(Socorro :: Symbols, defect)

Description

[Gabriele Svelto [:gsvelto] (PTO)]

The system-symbols-* tasks are currently broken even though some parts of them appear to be functioning. There's a few things that need to be done:

• Update the Docker images they're running on. They use some prehistoric Ubuntu version and we really want them on Debian 10
• Switch the Microsoft one to using the new dump_syms tool
• Convert the remaining Python 2 code to Python 3 including external tools

Some of this work overlaps bug 1603671 but we still need to do it here first since we need to get the tasks running again ASAP

Comment 1

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: Bug 1675263 - Refactor the scripts used to scrape Microsoft and Apple system symbols r=marco

This patch changes a few different things:

• The Docker image used for both scrapers are now based off of our standard
  Debian 10 image instead of an old Ubuntu one
• Both images were changed to make it easy to work with them when an
  interactive task is used
• The python packages used by the scripts had their versions pinned and
  cryptographic hashes have been recorded so that they can be verified before
  running the task
• The dump_syms version used on Windows was bumped to include fixes that allow
  us to scrape public symbols from DLLs and EXEs that have no PDB file available
• The maximum open file limit of the Window scraper was bumped up to cope with
  runs where we gather plenty of symbols in one go
• Similarly the maximum number of symultaneous connections to a single symbol
  server has been limited to 4 to avoid being throttled
• The macOS Python tools were modified to work with Python 3

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bae2af8ecf57
Refactor the scripts used to scrape Microsoft and Apple system symbols r=marco DONTBUILD

Comment 3

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/bae2af8ecf57

Comment 4

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr78/rev/f61863a946e2