962425 - legacy vcs-sync process doesn't auto retry when it should

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Hal Wine [:hwine] use NI!]

Should have been caught & corrected, but wasn't:

[[e-mail]3 ~]$ ps -flwwwwt pts/8
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
0 S vcs2vcs   9275  6671  0  80   0 - 27476 wait    2013 pts/8    00:00:00 /bin/
bash
0 S vcs2vcs  11472 14479  0  80   0 - 49145 sk_wai 03:44 pts/8    00:00:00 /opt/
vcs2vcs/venv/hg/bin/python /opt/vcs2vcs/bin/hg id http://hg.mozilla.org/releases/mozilla-b2g26_v1_2
0 S vcs2vcs  14479  9275  0  80   0 - 26524 pipe_w  2013 pts/8    00:02:22 /bin/bash /opt/vcs2vcs/bin/keep_updated
[[e-mail]3 ~]$ kill -hup 11472

Comment 1

[Hal Wine [:hwine] use NI!]

Another example - caused 6 hour delay to gecko.git updates:

[[e-mail]3 ~]$ ps flwwt pts/8
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
0   399 14142  2892  20   0 109904  3200 wait   Ss   pts/8      0:00 /bin/bash
0   399 17530 14142  20   0 106224  1396 pipe_w S+   pts/8      0:21  \_ /bin/bash /opt/vcs2vcs/bin/keep_updated
0   399 27467 17530  20   0 196576 12860 sk_wai S+   pts/8      0:00      \_ /opt/vcs2vcs/venv/hg/bin/python /opt/vcs2vcs/bin/hg id http://hg.mozilla.org/releases/mozilla-b2g28_v1_3t

Comment 2

[Hal Wine [:hwine] use NI!]

For historical reasons, keep_updated (gecko.git) uses a different logging method than do_update (all others). This means the test used by check_process_delay may not always identify the process to HUP.[1]

This could be a substantial change to legacy code -- deferring until it happens more often (boo!), or is replaced by the new system (yay!)

Comment 3

[Hal Wine [:hwine] use NI!]

Attachment: Also handle hangs from keep_updated

developed and worked on production, updating repo to that version

http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-tools/rev/6820a5aca6ba