1887940 - gecko-dev is not synchronizing since Mar 21, 2024

Status: RESOLVED FIXED

(Developer Services :: General, defect)

Description

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

Looking at https://github.com/mozilla/gecko-dev/commits/master/, the latest commit is on Mar 21, 2024 "Backed out changeset 968097672e64 (bug 1881076) for causing assertion failures @ gfx/layers/ipc/CanvasChild.cpp CLOSED TREE" (commit hash 2537e5eaafaab9a7ef6be5cfc8e9b1e2c747fdfd).

Comment 1

[:jmoss]

(In reply to Ting-Yu Lin [:TYLin] (PDT, UTC-7) from comment #0)

Looking at https://github.com/mozilla/gecko-dev/commits/master/, the latest commit is on Mar 21, 2024 "Backed out changeset 968097672e64 (bug 1881076) for causing assertion failures @ gfx/layers/ipc/CanvasChild.cpp CLOSED TREE" (commit hash 2537e5eaafaab9a7ef6be5cfc8e9b1e2c747fdfd).

Looking at this now.

Comment 2

[:jmoss]

It failed due to timeouts with connecting to hg.

17:57:03     INFO - retry: Calling run_command with args: [['/opt/vcs2vcs/vcssync1/build/venv/bin/hg', '--config', 'web.cacerts=/etc/pki/tls/certs/ca-bundle.crt', 'clone', '--noupdate', 'https://hg.mozilla.org/mozilla-central', '/opt/vcs2vcs/vcssync1/build/stage_source/mozilla-central']], kwargs: {'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fcdb3101618>, 'explanation': 'Automation Error: hg not responding', 'level': 'error'}, {'substr': 'unknown exception encountered', 'explanation': 'Automation Error: python exception in hg', 'level': 'error'}, {'substr': 'failed to import extension', 'explanation': 'Automation Error: hg extension missing', 'level': 'warning'}], 'output_timeout': 5700, 'cwd': '/opt/vcs2vcs/vcssync1/build'}, attempt #1
17:57:03     INFO - Running command: ['/opt/vcs2vcs/vcssync1/build/venv/bin/hg', '--config', 'web.cacerts=/etc/pki/tls/certs/ca-bundle.crt', 'clone', '--noupdate', 'https://hg.mozilla.org/mozilla-central', '/opt/vcs2vcs/vcssync1/build/stage_source/mozilla-central'] in /opt/vcs2vcs/vcssync1/build
17:57:03     INFO - Copy/paste: /opt/vcs2vcs/vcssync1/build/venv/bin/hg --config web.cacerts=/etc/pki/tls/certs/ca-bundle.crt clone --noupdate https://hg.mozilla.org/mozilla-central /opt/vcs2vcs/vcssync1/build/stage_source/mozilla-central
17:57:03     INFO - Calling ['/opt/vcs2vcs/vcssync1/build/venv/bin/hg', '--config', 'web.cacerts=/etc/pki/tls/certs/ca-bundle.crt', 'clone', '--noupdate', 'https://hg.mozilla.org/mozilla-central', '/opt/vcs2vcs/vcssync1/build/stage_source/mozilla-central'] with output_timeout 5700
17:57:04    ERROR -  abort: error: EOF occurred in violation of protocol (_ssl.c:590)
17:57:04    ERROR -  Automation Error: hg not responding
17:57:04    ERROR - Return code: 255

Comment 3

[:jmoss]

:glandium :sheehan

Did anything happen on March 21 (or around that time) that may have blocked this vcssync instance from connecting to hg? After restarting the instance, I'm still not seeing any updates and continue to see the errors shown in comment 2.

Comment 4

[Connor Sheehan [:sheehan]]

There were some issues that may have caused the original disconnect, but a restart should have resolved those. What is the IP address the machine is connecting from? It's possible I banned the IP/CIDR block of the machine.

Comment 5

[Connor Sheehan [:sheehan]]

Jonathan sent me the IP for vcs-sync and it was blocked in the load balancer. I've unblocked it and he has restarted the instance.

Comment 6

[:glob ✱]

Looks like this is still stuck.

Comment 7

[:jmoss]

I see the updates coming through now.

Comment 8

[Boris Chiou [:boris]]

Cool. I saw the new commits as well.

Comment 9

[Francesco Lodolo [:flod]]

Could it be that it's stuck again? If I understand correctly, gecko-dev is set up to sync frequently, and it's currently missing several commits from mozilla-central.