Bug 856691 (t-w864-ix-020)

t-w864-ix-020 problem tracking

RESOLVED FIXED

Status

Infrastructure & Operations
CIDuty
RESOLVED FIXED
5 years ago
2 months ago

People

(Reporter: bhearsum, Assigned: Q)

Tracking

Details

(Whiteboard: [buildduty])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
It's up but not connected to Buildbot. I can't access it via vnc or rdp. Reboot didn't help.

Updated

5 years ago
colo-trip: --- → scl3
(Reporter)

Comment 1

5 years ago
Whoops, I meant to file this in RelEng. Sorry!
Assignee: server-ops-dcops → nobody
Component: Server Operations: DCOps → Release Engineering: Machine Management
QA Contact: dmoore → armenzg
(Reporter)

Updated

5 years ago
Depends on: 857597
(Reporter)

Comment 2

5 years ago
Back in production after Q fixed some things up.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
https://tbpl.mozilla.org/php/getParsedLog.php?id=26091350&tree=Mozilla-Inbound

Burning reftests with:

14:39:47     INFO -  ###!!! ASSERTION: Desktop symbols failed to load.: 'Error', file e:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/gfx/gl/GLContext.cpp, line 329
14:39:47     INFO -  mozilla::gl::GLContextWGL::Init() [gfx/gl/GLContextProviderWGL.cpp:324]
14:39:47     INFO -  mozilla::gl::WGLLibrary::EnsureInitialized(bool) [gfx/gl/GLContextProviderWGL.cpp:246]
14:39:47     INFO -  mozilla::gl::GLContextProviderWGL::CreateOffscreen(nsIntSize const &,mozilla::gfx::SurfaceCaps const &,mozilla::gl::GLContext::ContextFlags) [gfx/gl/GLContextProviderWGL.cpp:663]
14:39:47     INFO -  mozilla::WebGLContext::SetDimensions(int,int) [content/canvas/src/WebGLContext.cpp:529]
14:39:47     INFO -  mozilla::dom::HTMLCanvasElement::UpdateContext(JSContext *,JS::Handle<JS::Value>) [content/html/content/src/HTMLCanvasElement.cpp:878]
14:39:47     INFO -  mozilla::dom::HTMLCanvasElement::GetContext(JSContext *,nsAString_internal const &,JS::Handle<JS::Value>,mozilla::ErrorResult &) [content/html/content/src/HTMLCanvasElement.cpp:805]
14:39:47     INFO -  mozilla::dom::HTMLCanvasElementBinding::getContext [obj-firefox/dom/bindings/HTMLCanvasElementBinding.cpp:148]
14:39:47     INFO -  mozilla::dom::HTMLCanvasElementBinding::genericMethod [obj-firefox/dom/bindings/HTMLCanvasElementBinding.cpp:563]
14:39:47     INFO -  js::CallJSNative(JSContext *,int (*)(JSContext *,unsigned int,JS::Value *),JS::CallArgs const &) [js/src/jscntxtinlines.h:225]
14:39:47     INFO -  js::Invoke(JSContext *,JS::CallArgs,js::MaybeConstruct) [js/src/vm/Interpreter.cpp:486]
14:39:47     INFO -  Interpret [js/src/vm/Interpreter.cpp:2514]
14:39:47     INFO -  js::RunScript(JSContext *,js::RunState &) [js/src/vm/Interpreter.cpp:443]
14:39:47     INFO -  js::Invoke(JSContext *,JS::CallArgs,js::MaybeConstruct) [js/src/vm/Interpreter.cpp:505]
14:39:47     INFO -  js::Invoke(JSContext *,JS::Value const &,JS::Value const &,unsigned int,JS::Value *,JS::MutableHandle<JS::Value>) [js/src/vm/Interpreter.cpp:536]
14:39:47     INFO -  JS_CallFunctionValue(JSContext *,JSObject *,JS::Value,unsigned int,JS::Value *,JS::Value *) [js/src/jsapi.cpp:5390]
14:39:47     INFO -  mozilla::dom::FrameRequestCallback::Call(JSContext *,JS::Handle<JSObject *>,double,mozilla::ErrorResult &) [obj-firefox/dom/bindings/WindowBinding.cpp:33]
14:39:47     INFO -  mozilla::dom::FrameRequestCallback::Call(double,mozilla::ErrorResult &,mozilla::dom::CallbackObject::ExceptionHandling) [obj-firefox/dist/include/mozilla/dom/WindowBinding.h:63]
14:39:47     INFO -  nsRefreshDriver::Tick(__int64,mozilla::TimeStamp) [layout/base/nsRefreshDriver.cpp:1141]
14:39:47     INFO -  mozilla::RefreshDriverTimer::TickDriver(nsRefreshDriver *,__int64,mozilla::TimeStamp) [layout/base/nsRefreshDriver.cpp:172]
14:39:47     INFO -  mozilla::RefreshDriverTimer::Tick() [layout/base/nsRefreshDriver.cpp:163]
14:39:47     INFO -  nsTimerImpl::Fire() [xpcom/threads/nsTimerImpl.cpp:544]
14:39:47     INFO -  nsTimerEvent::Run() [xpcom/threads/nsTimerImpl.cpp:630]
14:39:47     INFO -  nsThread::ProcessNextEvent(bool,bool *) [xpcom/threads/nsThread.cpp:622]
14:39:47     INFO -  NS_ProcessNextEvent(nsIThread *,bool) [obj-firefox/xpcom/build/nsThreadUtils.cpp:238]
14:39:47     INFO -  mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate *) [ipc/glue/MessagePump.cpp:81]
14:39:47     INFO -  MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:220]
14:39:47     INFO -  MessageLoop::RunHandler() [ipc/chromium/src/base/message_loop.cc:214]
14:39:47     INFO -  MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:188]
14:39:47     INFO -  nsBaseAppShell::Run() [widget/xpwidgets/nsBaseAppShell.cpp:165]
14:39:47     INFO -  nsAppShell::Run() [widget/windows/nsAppShell.cpp:113]
14:39:47     INFO -  nsAppStartup::Run() [toolkit/components/startup/nsAppStartup.cpp:270]
14:39:47     INFO -  XREMain::XRE_mainRun() [toolkit/xre/nsAppRunner.cpp:3858]
14:39:47     INFO -  XREMain::XRE_main(int,char * * const,nsXREAppData const *) [toolkit/xre/nsAppRunner.cpp:3926]
14:39:47     INFO -  XRE_main [toolkit/xre/nsAppRunner.cpp:4139]
14:39:47     INFO -  do_main [browser/app/nsBrowserApp.cpp:275]
14:39:47     INFO -  NS_internal_main(int,char * *) [browser/app/nsBrowserApp.cpp:635]
14:39:47     INFO -  wmain [toolkit/xre/nsWindowsWMain.cpp:112]
14:39:47     INFO -  __tmainCRTStartup [f:/dd/vctools/crt_bld/self_x86/crt/src/crtexe.c:552]
14:39:47     INFO -  wmainCRTStartup [f:/dd/vctools/crt_bld/self_x86/crt/src/crtexe.c:371]
14:39:47     INFO -  KERNEL32 + 0x28543
14:39:47     INFO -  ntdll + 0x5ac69
14:39:47     INFO -  ntdll + 0x5ac3c
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Disabled in slavealloc
Product: mozilla.org → Release Engineering
(Reporter)

Updated

5 years ago
Depends on: 913047
disabled again, not sure how it got re-enabled
(Reporter)

Comment 6

5 years ago
hardware diagnostics passed, not sure what to do

Updated

5 years ago
Depends on: 934511

Updated

5 years ago
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Depends on: 961214
on loan
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Reporter)

Updated

4 years ago
Depends on: 974039
(Reporter)

Comment 8

4 years ago
Back in production.
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago4 years ago
Resolution: --- → FIXED
Something stuck in its craw, making it hit the bug 981298 form of failing to remove files. Disabled in slavealloc.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attempting SSH reboot...Failed.
Attempting IPMI reboot...Failed.
Filed IT bug for reboot (bug 989208)

Comment 11

4 years ago
back online after reboot.

[vle@admin1a.private.scl3 ~]$ fping !$
fping t-w864-ix-020.wintest.releng.scl3.mozilla.com
t-w864-ix-020.wintest.releng.scl3.mozilla.com is alive
[vle@admin1a.private.scl3 ~]$ ssh !$
ssh t-w864-ix-020.wintest.releng.scl3.mozilla.com
The authenticity of host 't-w864-ix-020.wintest.releng.scl3.mozilla.com (10.26.40.50)' can't be established.
RSA key fingerprint is bc:e5:08:ac:f4:e9:2d:60:bb:5c:54:aa:6f:26:d1:ea.
Are you sure you want to continue connecting (yes/no)?
Noticed this slave was disabled - recent jobs are red.
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Depends on: 990569
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
back from reimage, rebooted
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Slave health looks awful on this, and I was tipped off to this by two consecutive failures on fx-team like this:
https://tbpl.mozilla.org/php/getParsedLog.php?id=38019863&tree=Fx-Team

Disabled.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
bug 990569 comment 6 (and the general smell of the failures) makes me wonder whether it's using the wrong video card at the wrong resolution.

Comment 16

4 years ago
Q, this machine is one that would display the "keep changes" dialog.

I've also run fakemon.vbs on a "good state" and it would still try to make changes.

Let me a attach a screen share.

Comment 17

4 years ago
Created attachment 8415450 [details]
video.ogv
(Assignee)

Comment 18

4 years ago
For some reason the screen order is wrong the nvidia card is coming up as the primary device. It is possible something in wrong in the bios
Disabled in slavealloc, since it's still running production jobs, and still failing them in odd ways.

Updated

4 years ago
Depends on: 1011681

Comment 20

4 years ago
Asked IT to look at the graphical setup.
QA Contact: armenzg → bugspam.Callek
Rebooted into production
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Still busted: bug 977561 and bug 1003614 are actually honeypots for Win8 slaves running at 1024x768, which this one is.

Disabled in slavealloc.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Depends on: 1004813
(Assignee)

Comment 23

4 years ago
Fixed via Bug 1091708
Assignee: nobody → q
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Depends on: 1198420
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Re imaged and enabled in slavealloc
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago3 years ago
Resolution: --- → FIXED
Graphics are busted, see the rate of failure in at least web-platform-tests-1, web-platform-tests-3 and mochitest-2 in https://secure.pub.build.mozilla.org/buildapi/recent/t-w864-ix-020?numbuilds=500. It appears to be pretty much unable to play video successfully. Disabled.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Still busted, redisabled.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED

Updated

2 months ago
Product: Release Engineering → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.