1657392 - Probe scraper failing on Firefox Desktop Glean App

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect, P1)

Description

[Frank Bertsch [:frank]]

Error:

 "Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/probe_scraper/runner.py", line 449, in <module>
    main(args.cache_dir,
  File "/app/probe_scraper/runner.py", line 377, in main
    load_glean_metrics(cache_dir, out_dir, repositories_file, dry_run, glean_repo)
  File "/app/probe_scraper/runner.py", line 208, in load_glean_metrics
    commit_timestamps, repos_metrics_data, emails = git_scraper.scrape(cache_dir, repositories)
  File "/app/probe_scraper/scrapers/git_scraper.py", line 139, in scrape
    ts, commits = retrieve_files(repo_info, folder)
  File "/app/probe_scraper/scrapers/git_scraper.py", line 84, in retrieve_files
    contents = get_file_at_hash(repo, _hash, rel_path)
  File "/app/probe_scraper/scrapers/git_scraper.py", line 52, in get_file_at_hash
    return repo.git.show("{hash}:{path}".format(hash=_hash, path=filename))
  File "/usr/local/lib/python3.8/site-packages/git/cmd.py", line 542, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/git/cmd.py", line 1005, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.8/site-packages/git/cmd.py", line 822, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)

stderr: 'fatal: Path 'toolkit/components/glean/pings.yaml' exists on disk, but not in '4520632fe0664572c5f70688595b7721d167e2d0'.'

Comment 1

[Frank Bertsch [:frank]]

Looks like pings.yaml was backed out in this change, which causes us to fail in the probe-scraper.

I'll only scrape after that date, similar to what we did for breaking changes to the format.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/probe-scraper/pull/213

Comment 3

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub master push: https://github.com/mozilla/probe-scraper/commit/3a8578be154fbfbdcf18d0cc995288c9c47f635b

Commit merged into master by GitHub
Authored by Frank Bertsch (fbertsch)