Investigate talos-r4-snow-053 (it's fine), then setup after reimaging

RESOLVED FIXED

Status

Release Engineering
General
P3
normal
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: nthomas, Assigned: coop)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [buildduty][buildslaves])

(Reporter)

Description

6 years ago
talos-r4-snow-053 got rebooted last time (bug 707234), did 20 jobs OK over about 5 hours, then started having issues writing to the disk.

eg, at the end of a talos dromaeo test:
Traceback (most recent call last):
  File "bcontroller.py", line 242, in <module>
    sys.exit(main())
  File "bcontroller.py", line 239, in main
    bcontroller.run()
  File "bcontroller.py", line 206, in run
    results_file.close()
IOError: [Errno 6] Device not configured
Traceback (most recent call last):
  File "run_tests.py", line 596, in <module>
    main()
  File "run_tests.py", line 593, in main
    test_file(arg, options.screen, options.amo)
  File "run_tests.py", line 521, in test_file
    browser_dump, counter_dump, print_format = mytest.runTest(browser_config, test)
  File "/Users/cltbld/talos-slave/talos-data/talos/ttest.py", line 424, in runTest
    results_raw = results_file.read()
IOError: [Errno 6] Device not configured

and then we tried to call our reboot script:

Traceback (most recent call last):
  File "count_and_reboot.py", line 55, in <module>
    if increment_count(options.countfile) >= options.maxcount:
  File "count_and_reboot.py", line 36, in increment_count
    open(fname, "w").write("%i\n" % current_count)
IOError: [Errno 22] invalid mode ('w') or filename: '../talos_count.txt'

Please reboot to an external disk and run a disk check on this mini.
Assignee: server-ops-releng → jwatkins
colo-trip: --- → scl1
It passed both a short hardware test (via diag cd) and a disk utility test (via netboot).  Since I didn't find any errors in either test, I re-imaged the disk.  If this mini continues to fail,  reopen this bug and we will bring it in for repairs.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
(Reporter)

Comment 2

6 years ago
Ok, thanks for looking. We'll get this back into production.
Assignee: jwatkins → nobody
Status: RESOLVED → REOPENED
Component: Server Operations: RelEng → Release Engineering
QA Contact: mrz → release
Resolution: FIXED → ---
Summary: Investigate talos-r4-snow-053 → Investigate talos-r4-snow-053 (it's fine), then setup after reimaging
Whiteboard: [buildduty][buildslaves]
(Assignee)

Updated

6 years ago
Assignee: nobody → coop
Priority: -- → P3
(Assignee)

Comment 3

6 years ago
Back in production.
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.