Closed Bug 1475883 Opened 6 years ago Closed 6 years ago

f97ba5c963ffbda5cb0b793047e581ae55d20530 missing in the hg -> git map

Categories

(Developer Services :: Git, enhancement)

Product:
Developer Services
Component:
Git
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: marco, Unassigned)

References

Details

The sync process log shows that it failed. I'm checking into why, and to fix it.
Flags: needinfo?(dhouse)
The last sync for beagle was on Saturday, July 14, at 23:53, and it completed with no problems.

The failure is happening in the pull from mozharness. When I manually try the pull, I get a hanging state and I did not leave it to time-out/fail for more than one minute (same traceback for my KeyboardInterrupt as for the SigInt in the sync's failed log):

```
[vcs2vcs@ip-172-31-30-125 vcssync1]$ cat beagle.log
2018-07-16 13:41:01 pid-26999 Acquired lock
2018-07-16 13:41:01 pid-26999 no missed locks
2018-07-16 13:41:01 pid-26999 Updating mozharness
Traceback (most recent call last):
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 97, in _runcatch
    return _dispatch(req)
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 778, in _dispatch
    cmdpats, cmdoptions)
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 549, in runcommand
    ret = _runcommand(ui, options, cmd, d)
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 869, in _runcommand
    return checkargs()
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 840, in checkargs
    return cmdfunc()
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 775, in <lambda>
    d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
  File "/usr/lib64/python2.7/dist-packages/mercurial/util.py", line 506, in check
    return func(*args, **kwargs)
  File "/usr/lib64/python2.7/dist-packages/mercurial/extensions.py", line 143, in wrap
    util.checksignature(origfn), *args, **kwargs)
  File "/usr/lib64/python2.7/dist-packages/mercurial/util.py", line 506, in check
    return func(*args, **kwargs)
  File "/usr/lib64/python2.7/dist-packages/hgext/mq.py", line 3528, in mqcommand
    return orig(ui, repo, *args, **kwargs)
  File "/usr/lib64/python2.7/dist-packages/mercurial/util.py", line 506, in check
    return func(*args, **kwargs)
  File "/usr/lib64/python2.7/dist-packages/mercurial/commands.py", line 4537, in pull
    other = hg.peer(repo, opts, source)
  File "/usr/lib64/python2.7/dist-packages/mercurial/hg.py", line 122, in peer
    return _peerorrepo(rui, path, create).peer()
  File "/usr/lib64/python2.7/dist-packages/mercurial/hg.py", line 102, in _peerorrepo
    obj = _peerlookup(path).instance(ui, path, create)
  File "/usr/lib64/python2.7/dist-packages/mercurial/httppeer.py", line 238, in instance
    inst._fetchcaps()
  File "/usr/lib64/python2.7/dist-packages/mercurial/httppeer.py", line 57, in _fetchcaps
    self.caps = set(self._call('capabilities').split())
  File "/usr/lib64/python2.7/dist-packages/mercurial/httppeer.py", line 171, in _call
    fp = self._callstream(cmd, **args)
  File "/usr/lib64/python2.7/dist-packages/mercurial/httppeer.py", line 118, in _callstream
    resp = self.urlopener.open(req)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/dist-packages/mercurial/url.py", line 315, in http_open
    return self.do_open(httpconnection, req)
  File "/usr/lib64/python2.7/dist-packages/mercurial/keepalive.py", line 255, in do_open
    r = h.getresponse()
  File "/usr/lib64/python2.7/dist-packages/mercurial/url.py", line 190, in getresponse
    return keepalive.HTTPConnection.getresponse(self)
  File "/usr/lib64/python2.7/dist-packages/mercurial/keepalive.py", line 577, in safegetresponse
    return cls.getresponse(self)
  File "/usr/lib64/python2.7/httplib.py", line 1132, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib64/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib64/python2.7/dist-packages/mercurial/dispatch.py", line 78, in catchterm
    raise error.SignalInterrupt
SignalInterrupt
killed!
```

```
[...]
  File "/usr/lib64/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib64/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
KeyboardInterrupt
interrupted!
```
I switched the mozharness clone to https from http. Using http, a clone/pull hangs so we may be requiring https/ssl for hgmo now.

The sync is running. I'll check when it completes if these mappings are still missing and need manually pushed.
3/5 mapped so far:
```
[vcs2vcs@ip-172-31-30-125 ~]$ grep 'f97ba5c963ffbda5cb0b793047e581ae55d20530\|2a8f94a45fd3f898cb63c54b8df662f8ad1f7527\|315103f6db6b1f44647d82c0f494c2d45fc42e07\|6024634afa8c90d56d2025d46f33d8bf661abab1\w|6a320851d377068d46a59ff11d5d5124b219138a' ./vcssync1/build/conversion/beagle/.hg/git-mapfile
14976cd6b38abf60317c5c676d7f0c55ceea2321 2a8f94a45fd3f898cb63c54b8df662f8ad1f7527
10d4df5aa951f9e1b58325e110b88d6c69e38e8e 315103f6db6b1f44647d82c0f494c2d45fc42e07
1261098dca8b8ff6e183a6778e06ec018551c6d8 f97ba5c963ffbda5cb0b793047e581ae55d20530
[vcs2vcs@ip-172-31-30-125 ~]$ date
Mon Jul 16 14:25:42 UTC 2018
```
All 5/5 are mapped in the sync server and Vcssync lists all 5 in published-to-mapper,
```
[vcs2vcs@ip-172-31-30-125 ~]$ grep 'f97ba5c963ffbda5cb0b793047e581ae55d20530\|2a8f94a45fd3f898cb63c54b8df662f8ad1f7527\|315103f6db6b1f44647d82c0f494c2d45fc42e07\|6024634afa8c90d56d2025d46f33d8bf661abab1\|6a320851d377068d46a59ff11d5d5124b219138a' ./vcssync1/logs/published*

14976cd6b38abf60317c5c676d7f0c55ceea2321 2a8f94a45fd3f898cb63c54b8df662f8ad1f7527
10d4df5aa951f9e1b58325e110b88d6c69e38e8e 315103f6db6b1f44647d82c0f494c2d45fc42e07
78b9d7d52c388420734f186507ce4f5eed68916c 6024634afa8c90d56d2025d46f33d8bf661abab1
2a8b467320b2d761256911b369015dd988ab1a1b 6a320851d377068d46a59ff11d5d5124b219138a
1261098dca8b8ff6e183a6778e06ec018551c6d8 f97ba5c963ffbda5cb0b793047e581ae55d20530
```

And after 20 minutes, they showed up in the mapper (the publish may have been delayed by the sync catching-up from missing the last day):
```
[vcs2vcs@ip-172-31-30-125 ~]$ for I in f97ba5c963ffbda5cb0b793047e527 315103f6db6b1f44647d82c0f494c2d45fc42e07 6024634afa8c90d56d2025d46f33d8bf661abab1 6a320851d377068d46a59ff11d5d5124b219138a; do wget -O - -q https://mapper.mozilla-releng.net/gecko-dev/rev/hg/$I; echo; done
1261098dca8b8ff6e183a6778e06ec018551c6d8 f97ba5c963ffbda5cb0b793047e581ae55d20530
14976cd6b38abf60317c5c676d7f0c55ceea2321 2a8f94a45fd3f898cb63c54b8df662f8ad1f7527
10d4df5aa951f9e1b58325e110b88d6c69e38e8e 315103f6db6b1f44647d82c0f494c2d45fc42e07
78b9d7d52c388420734f186507ce4f5eed68916c 6024634afa8c90d56d2025d46f33d8bf661abab1
2a8b467320b2d761256911b369015dd988ab1a1b 6a320851d377068d46a59ff11d5d5124b219138a
```
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
I also moved the project-branches sync to use https for mozharness. The build-repos were already setup with https for mozharness, and the l10n sync is using git://github.com/mozilla/build-mozharness (which is synced by build-repos).
See Also: → 1521038
See Also: → 1551348
See Also: → 1553068
See Also: → 1555175
See Also: → 1568507
See Also: → 1569277
See Also: → 1595816
You need to log in before you can comment on or make changes to this bug.