Closed Bug 1024047 Opened 10 years ago Closed 10 years ago

Intermittent build.test.js | Node modules tests make node_modules from github or git mirror

Categories

(Firefox OS Graveyard :: Gaia::Build, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: RyanVM, Assigned: rickychien)

References

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

https://tbpl.mozilla.org/php/getParsedLog.php?id=41536811&tree=B2g-Inbound b2g_ubuntu64_vm b2g-inbound opt test gaia-build on 2014-06-11 10:38:10 PDT for push 618a4eb46fda slave: tst-linux64-spot-418 10:51:26 INFO - TEST-START | Node modules tests make node_modules from git mirror 10:53:35 INFO - stdout: make[1]: Entering directory `/builds/slave/test/gaia' 10:53:35 INFO - if [ ! -d "git-gaia-node-modules" ] ; then \ 10:53:35 INFO - git clone "https://git.mozilla.org/b2g/gaia-node-modules.git" "git-gaia-node-modules" ; \ 10:53:35 INFO - fi 10:53:35 INFO - Cloning into 'git-gaia-node-modules'... 10:53:35 INFO - make[1]: Leaving directory `/builds/slave/test/gaia' 10:53:35 INFO - stderr: error: Failed connect to git.mozilla.org:443; Connection timed out while accessing https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs 10:53:35 INFO - fatal: HTTP request failed 10:53:35 INFO - make[1]: *** [git-gaia-node-modules] Error 128 10:53:35 INFO - error: Error: Command failed: error: Failed connect to git.mozilla.org:443; Connection timed out while accessing https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs 10:53:35 INFO - fatal: HTTP request failed 10:53:35 INFO - make[1]: *** [git-gaia-node-modules] Error 128 10:53:35 INFO - TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/build/test/integration/build.test.js | Node modules tests make node_modules from git mirror 10:53:35 INFO - TEST-END | Node modules tests make node_modules from git mirror
Jonathan, I think you probably encountered such issues in the past, do you know what could go wrong here? Should we report this to IT?
Flags: needinfo?(jgriffin)
Flags: needinfo?(yurenju.mozilla)
Hal, can you think of any reason why we might be getting: 10:53:35 INFO - stderr: error: Failed connect to git.mozilla.org:443; Connection timed out while accessing https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs while cloning a git.mozilla.org repo from a buildbot job? Is git.mozilla.org overloaded again due to the increased traffic we're seeing from gaia-try?
Flags: needinfo?(jgriffin) → needinfo?(hwine)
(In reply to Jonathan Griffin (:jgriffin) from comment #5) > Hal, can you think of any reason why we might be getting: > > 10:53:35 INFO - stderr: error: Failed connect to git.mozilla.org:443; > Connection timed out while accessing > https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs No, but I'll note that most of the cases posted to this bug to date are about failures from github, as I read the logs. (Updated summary to reflect that) That would seem to eliminate git.m.o as part of the issue. > > while cloning a git.mozilla.org repo from a buildbot job? Is > git.mozilla.org overloaded again due to the increased traffic we're seeing > from gaia-try? We're not aware of any performance issues - none show on any of the monitoring.
Flags: needinfo?(hwine)
Summary: Intermittent build.test.js | Node modules tests make node_modules from git mirror → Intermittent build.test.js | Node modules tests make node_modules from github or git mirror
Flags: needinfo?(yurenju.mozilla)
Yuren, it looks like the tests are trying to connect to github. But I don't get why we don't always get the error. To me, it would be either always or never :) I don't know if you can make some sense out of this?
Flags: needinfo?(yurenju.mozilla)
Also note that tests hitting the outside network is a recipe for intermittent failures. It's why we're explicitly clamping down on such things.
I don't know, but is that possible if github ban us temporally when we reach some bandwith quota? I also contacted github to see if this is an issue from there.
github made some infrastructure updates so let's wait further log from try server.
Assignee: nobody → yurenju.mozilla
Flags: needinfo?(yurenju.mozilla)
we can add manifest for build system just like gaia integration test and disable github test on tbpl, so it should be two part of this issue: 1. add manifest mechanism in gaia build system 2. modify mozharness script to add BUILD_TEST_MANIFEST for Gb just like TEST_MANIFEST for gaia integration test.
Comment on attachment 8487026 [details] [review] github PR: https://github.com/mozilla-b2g/gaia/pull/23894 I think this patch looks fine. I left just few questions on github. r=gduan
Attachment #8487026 - Flags: review?(gduan) → review+
bug 1072842 is filed for ading BUILD_TEST_MANIFEST to try server.
Attached file Gaia PR
Blocks: 1059449
Assignee: yurenju → ricky060709
No longer blocks: 1059449
Status: NEW → ASSIGNED
Comment on attachment 8540039 [details] [review] Gaia PR PR completed. In this patch: 1. I rename all build tests from .test.js to _test.js 2. To disable specific tests, using mocha "--invert --grep (-ig) pattern" to exclude tests we want so that we don't need to skip files by creating manifest / blacklist.
Attachment #8540039 - Attachment description: WIP PR → Gaia PR
Attachment #8540039 - Flags: review?(gduan)
Blocks: 1059449
After landing this bug, we'll add TRY_ENV=1 in try server to disable network connections in follow-up bug 1072842.
Comment on attachment 8540039 [details] [review] Gaia PR Great job! r=gduan. As comment 47, your patch will filter the test containing "Network Required" key words if setting TRY_ENV=1.
Attachment #8540039 - Flags: review?(gduan) → review+
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: