Closed Bug 993632 Opened 10 years ago Closed 10 years ago

gaia github->git.m.o sync is broken

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jlund, Unassigned)

References

Details

Attachments

(1 file)

      No description provided.
checking on bm66

/builds/b2g_bumper/b2g_bumper.log and /builds/gaia_bumper/gaia_bumper.log both have recent output picking up revisions and I can not see any error output.

aki is landing a patch that may help debugging status of the bumper though.
Attached patch log rotateSplinter Review
Attachment #8403520 - Flags: review?(jlund)
looking at procs I am getting varied results:

[cltbld@buildbot-master66.srv.releng.usw2.mozilla.com b2g_bumper]$ ps -elfy | grep bumper
S cltbld    8429 23396  0  80   0   844 25825 pipe_w 12:54 pts/0    00:00:00 grep bumper

[cltbld@buildbot-master66.srv.releng.usw2.mozilla.com b2g_bumper]$ ps -elfy | grep bumper
S cltbld    8464  8462  0  80   0  1244  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8465  8463  0  80   0  1204  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_gaia_bumper.sh
S cltbld    8488  8465  1  80   0 10140 24252 pipe_w 12:55 ?        00:00:00 python mozharness/scripts/bump_gaia_json.py -c gaia_bumper/gaia_json.py
S cltbld    8494  8464  0  80   0   572  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8496  8494  1  80   0 11796 27161 pipe_w 12:55 ?        00:00:00 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/master.py
S cltbld    8497  8464  0  80   0   576  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8499  8497  1  80   0 11792 27159 pipe_w 12:55 ?        00:00:00 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/v1.4.py
S cltbld    8500  8464  0  80   0   576  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8502  8500  2  80   0 11788 27159 pipe_w 12:55 ?        00:00:00 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/v1.3.py
S cltbld    8503  8464  0  80   0   576  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8504  8503  1  80   0 11736 27096 pipe_w 12:55 ?        00:00:00 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/v1.3t.py
S cltbld    8506  8504  0  80   0  8380 20243 wait   12:55 ?        00:00:00 python /usr/local/bin/hgtool.py --bundle https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles/mozilla-b2g28_v1_3t.hg https://hg.mozilla.org/releases/mozilla-b2g28_v1_3t /builds/b2g_bumper/v1.3t/build/mozilla-b2g28_v1_3t
S cltbld    8510  8502  0  80   0  8380 20243 wait   12:55 ?        00:00:00 python /usr/local/bin/hgtool.py --bundle https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles/mozilla-b2g28_v1_3.hg https://hg.mozilla.org/releases/mozilla-b2g28_v1_3 /builds/b2g_bumper/v1.3/build/mozilla-b2g28_v1_3
S cltbld    8511  8496  1  80   0  8384 20245 wait   12:55 ?        00:00:00 python /usr/local/bin/hgtool.py --bundle https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles/b2g-inbound.hg https://hg.mozilla.org/integration/b2g-inbound /builds/b2g_bumper/master/build/b2g-inbound
S cltbld    8512  8499  1  80   0  8380 20243 wait   12:55 ?        00:00:00 python /usr/local/bin/hgtool.py --bundle https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles/mozilla-aurora.hg https://hg.mozilla.org/releases/mozilla-aurora /builds/b2g_bumper/v1.4/build/mozilla-aurora
S cltbld    8537 23396  0  80   0   848 25826 pipe_w 12:55 pts/0    00:00:00 grep bumper


[cltbld@buildbot-master66.srv.releng.usw2.mozilla.com b2g_bumper]$ ps -elfy | grep bumper
S cltbld    8464  8462  0  80   0  1244  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8465  8463  0  80   0  1204  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_gaia_bumper.sh
S cltbld    8488  8465  0  80   0 11504 26122 pipe_w 12:55 ?        00:00:00 python mozharness/scripts/bump_gaia_json.py -c gaia_bumper/gaia_json.py
S cltbld    8494  8464  0  80   0   572  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
R cltbld    8496  8494  5  80   0 17508 332314 -     12:55 ?        00:00:03 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/master.py
S cltbld    8497  8464  0  80   0   576  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8499  8497  5  80   0 17672 332055 futex_ 12:55 ?       00:00:03 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/v1.4.py
S cltbld    8500  8464  0  80   0   576  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8502  8500  5  80   0 17556 332049 futex_ 12:55 ?       00:00:03 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/v1.3.py
S cltbld    8503  8464  0  80   0   576  2322 wait   12:55 ?        00:00:00 /bin/bash /usr/local/bin/run_b2g_bumper.sh
S cltbld    8504  8503  1  80   0 13784 281879 pipe_w 12:55 ?       00:00:01 python ../mozharness/scripts/b2g_bumper.py -c b2g_bumper/v1.3t.py
R cltbld    9797  8504  6  80   0 82420 65937 -      12:55 ?        00:00:02 /tools/python27/bin/python2.7 /usr/local/bin/hg --config web.cacerts=/etc/pki/tls/certs/ca-bundle.crt commit -u B2G Bumper Bot <release+b2gbumper@mozilla.com> -m Bumping manifests a=b2g-bump
S cltbld   11149 23396  0  80   0   864 25826 pipe_w 12:56 pts/0    00:00:00 grep bumper
Comment on attachment 8403520 [details] [diff] [review]
log rotate

so I see bump_gaia_json.py uses this: http://mxr.mozilla.org/build/source/mozharness/configs/gaia_bumper/gaia_json.py#6

but looks like that's not the config used for b2g_bumper.py

so lgtm more backups!
Attachment #8403520 - Flags: review?(jlund) → review+
Comment on attachment 8403520 [details] [diff] [review]
log rotate

http://hg.mozilla.org/build/mozharness/rev/379a327473c1

These will show up in /builds/b2g_bumper/*/build/upload/logs
Attachment #8403520 - Flags: checked-in+
For some reason, we can no longer connect to github from the conversion box:

[vcs2vcs@github-sync3.dmz.scl3 tmp]$ nc -vz github.com 9418
nc: connect to github.com port 9418 (tcp) failed: Connection timed out
[vcs2vcs@github-sync3.dmz.scl3 tmp]$
Summary: bumper bot may be broken. b-i, aurora, b2g28 had gaia commits and are not firing jobs in tbpl → gaia github->git.m.o sync is broken
"github is working on changing routing preferences on their end to route our traffic to ntt instead of L3 in hopes that will fix the problem with connectivity to them"

"Mozilla has escalated the issue to above.net and we're attempting to resolve the issue ourselves by modifying our routing policies."
Depends on: 993766
We applied a temporary fix to solve comment 6 (and everything in .dmz.scl3) the time this issue is fixed.
https://bugzilla.mozilla.org/show_bug.cgi?id=993766#c22 and followup comments.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Component: Tools → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: