Closed Bug 1024047 Opened 10 years ago Closed 10 years ago

Intermittent build.test.js | Node modules tests make node_modules from github or git mirror

Categories

(Firefox OS Graveyard :: Gaia::Build, defect)

Product:
Firefox OS Graveyard
Component:
Gaia::Build
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: RyanVM, Assigned: rickychien)

References

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

github PR: https://github.com/mozilla-b2g/gaia/pull/23894
46 bytes, text/x-github-pull-request
gduan
: review+
Details | Review
Gaia PR
46 bytes, text/x-github-pull-request
gduan
: review+
Details | Review 
https://tbpl.mozilla.org/php/getParsedLog.php?id=41536811&tree=B2g-Inbound

b2g_ubuntu64_vm b2g-inbound opt test gaia-build on 2014-06-11 10:38:10 PDT for push 618a4eb46fda
slave: tst-linux64-spot-418

10:51:26     INFO -  TEST-START | Node modules tests make node_modules from git mirror
10:53:35     INFO -  stdout: make[1]: Entering directory `/builds/slave/test/gaia'
10:53:35     INFO -  if [ ! -d "git-gaia-node-modules" ] ; then \
10:53:35     INFO -  		git clone "https://git.mozilla.org/b2g/gaia-node-modules.git" "git-gaia-node-modules" ; \
10:53:35     INFO -  	fi
10:53:35     INFO -  Cloning into 'git-gaia-node-modules'...
10:53:35     INFO -  make[1]: Leaving directory `/builds/slave/test/gaia'
10:53:35     INFO -  stderr: error: Failed connect to git.mozilla.org:443; Connection timed out while accessing https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs
10:53:35     INFO -  fatal: HTTP request failed
10:53:35     INFO -  make[1]: *** [git-gaia-node-modules] Error 128
10:53:35     INFO -  error: Error: Command failed: error: Failed connect to git.mozilla.org:443; Connection timed out while accessing https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs
10:53:35     INFO -  fatal: HTTP request failed
10:53:35     INFO -  make[1]: *** [git-gaia-node-modules] Error 128
10:53:35     INFO -  TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/build/test/integration/build.test.js | Node modules tests make node_modules from git mirror
10:53:35     INFO -  TEST-END | Node modules tests make node_modules from git mirror
Jonathan, I think you probably encountered such issues in the past, do you know what could go wrong here?

Should we report this to IT?
Flags: needinfo?(jgriffin)
Flags: needinfo?(yurenju.mozilla)
Hal, can you think of any reason why we might be getting:

10:53:35     INFO -  stderr: error: Failed connect to git.mozilla.org:443; Connection timed out while accessing https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs

while cloning a git.mozilla.org repo from a buildbot job?  Is git.mozilla.org overloaded again due to the increased traffic we're seeing from gaia-try?
Flags: needinfo?(jgriffin) → needinfo?(hwine)
(In reply to Jonathan Griffin (:jgriffin) from comment #5)
> Hal, can you think of any reason why we might be getting:
> 
> 10:53:35     INFO -  stderr: error: Failed connect to git.mozilla.org:443;
> Connection timed out while accessing
> https://git.mozilla.org/b2g/gaia-node-modules.git/info/refs

No, but I'll note that most of the cases posted to this bug to date are about failures from github, as I read the logs. (Updated summary to reflect that)

That would seem to eliminate git.m.o as part of the issue.

> 
> while cloning a git.mozilla.org repo from a buildbot job?  Is
> git.mozilla.org overloaded again due to the increased traffic we're seeing
> from gaia-try?

We're not aware of any performance issues - none show on any of the monitoring.
Flags: needinfo?(hwine)
Summary: Intermittent build.test.js | Node modules tests make node_modules from git mirror → Intermittent build.test.js | Node modules tests make node_modules from github or git mirror
Flags: needinfo?(yurenju.mozilla)
Yuren, it looks like the tests are trying to connect to github. But I don't get why we don't always get the error. To me, it would be either always or never :)

I don't know if you can make some sense out of this?
Flags: needinfo?(yurenju.mozilla)
Also note that tests hitting the outside network is a recipe for intermittent failures. It's why we're explicitly clamping down on such things.
I don't know, but is that possible if github ban us temporally when we reach some bandwith quota?

I also contacted github to see if this is an issue from there.
github made some infrastructure updates so let's wait further log from try server.
Assignee: nobody → yurenju.mozilla
Flags: needinfo?(yurenju.mozilla)
we can add manifest for build system just like gaia integration test and disable github test on tbpl, so it should be two part of this issue:

1. add manifest mechanism in gaia build system
2. modify mozharness script to add BUILD_TEST_MANIFEST for Gb just like TEST_MANIFEST for gaia integration test.
Comment on attachment 8487026 [details] [review]
github PR: https://github.com/mozilla-b2g/gaia/pull/23894

I think this patch looks fine. I left just few questions on github. r=gduan
Attachment #8487026 - Flags: review?(gduan) → review+
bug 1072842 is filed for ading BUILD_TEST_MANIFEST to try server.
Attached file Gaia PR — 
Blocks: 1059449
Assignee: yurenju → ricky060709
No longer blocks: 1059449
Status: NEW → ASSIGNED

    
    

    
  
Comment on attachment 8540039 [details] [review]
Gaia PR

PR completed.

In this patch:

1. I rename all build tests from .test.js to _test.js
2. To disable specific tests, using mocha "--invert --grep (-ig) pattern" to exclude tests we want so that we don't need to skip files by creating manifest / blacklist.

        Attachment #8540039 -
        Attachment description: WIP PR → Gaia PR

        Attachment #8540039 -
        Flags: review?(gduan)
Blocks: 1059449

    
    

    
  
After landing this bug, we'll add TRY_ENV=1 in try server to disable network connections in follow-up bug 1072842.

    
    

    
  
Comment on attachment 8540039 [details] [review]
Gaia PR

Great job! r=gduan.

As comment 47, your patch will filter the test containing "Network Required" key words if setting TRY_ENV=1.

        Attachment #8540039 -
        Flags: review?(gduan) → review+

    
    

    
  
Thanks!

Merged in master: https://github.com/mozilla-b2g/gaia/commit/85c3579e91d99be47665e1e1d61f20e548cb04cb
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED

    
    

    
      
  

    
    

    
      
  

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: