1919464 - Coverage aggregations tasks are failing with unknown errors

Status: NEW

(Testing :: Code Coverage, defect)

Description

[Marco Castelluccio [:marco]]

2024-09-18 04:36:07.000067 [INFO    ] base.py: Building covdir suite report
2024-09-18 04:40:12.000232 [ERROR   ] utils.py: Command failed with code: -6
2024-09-18 04:40:12.000238 [ERROR   ] grcov.py: Error while running grcov
Traceback (most recent call last):
  File "/usr/local/bin/code-coverage-repo", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/hooks/repo.py", line 242, in main
    hook.run()
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/hooks/repo.py", line 117, in run
    reports = self.build_reports(only=[("all", "all")])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/hooks/base.py", line 148, in build_reports
    output = grcov.report(
             ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/grcov.py", line 30, in report
    return run_check(cmd)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/utils.py", line 66, in run_check
    raise Exception(f"`{command[0]}` failed with code: {proc.returncode}.")
Exception: `grcov` failed with code: -6.

Comment 1

[BMO Github Automation]

Attachment: [mozilla-releng/fxci-config] Bug 1919464 - Bump code coverage workers to c2-standard-8 to avoid OOMs (#101)

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:marco, could you have a look please?

For more information, please visit BugBot documentation.

Comment 3

[BMO Github Automation]

Attachment: [mozilla-releng/fxci-config] Bug 1919464 - bump code-coverage/bot-gcp instance type to c2-standard-16 (#166)

Comment 4

[Julien Cristau [:jcristau]]

It seems this didn't help, grcov still ends up getting killed (https://firefox-ci-tc.services.mozilla.com/tasks/EfNOpuJHQ5-r1AzsK0TuaA/runs/0/logs/public/logs/live_backing.log e.g.).

2024-11-11 14:13:22.000047 [ERROR   ] grcov.py: Error while running grcov
Traceback (most recent call last):
  File "/usr/local/bin/code-coverage-repo", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/hooks/repo.py", line 242, in main
    hook.run()
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/hooks/repo.py", line 117, in run
    reports = self.build_reports(only=[("all", "all")])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/hooks/base.py", line 148, in build_reports
    output = grcov.report(
             ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/grcov.py", line 30, in report
    return run_check(cmd)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/code_coverage_bot/utils.py", line 68, in run_check
    raise Exception(f"`{command[0]}` failed with code: {proc.returncode}.")
Exception: `grcov` failed with code: -9.
Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit
[taskcluster 2024-11-11 14:13:24.888Z] === Task Finished ===
[taskcluster 2024-11-11 14:13:24.948Z] Unsuccessful task run with exit code: 1 completed in 664.047 seconds