Closed Bug 922042 Opened 8 years ago Closed 8 years ago

android_panda_talos_releng.py is still using build.m.o

Categories

(Release Engineering :: General, defect)

ARM
Android
defect
Not set
blocker

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: emorley)

References

Details

Attachments

(1 file)

Now build.m.o has been switched off, Android 4.0 talos jobs are failing with:

https://tbpl.mozilla.org/php/getParsedLog.php?id=28551612&tree=Mozilla-Inbound#error0

{
04:45:12    FATAL - Can't download from http://build.mozilla.org/talos/zips/retry.zip to /builds/panda-0786/test/build/retry.zip!
04:45:12    FATAL - Caught exception: HTTP Error 403: Forbidden
04:45:12    FATAL - Caught exception: HTTP Error 403: Forbidden
04:45:12    FATAL - Caught exception: HTTP Error 403: Forbidden
04:45:12    FATAL - Caught exception: HTTP Error 403: Forbidden
04:45:12    FATAL - Caught exception: <urlopen error [Errno -5] No address associated with hostname>
04:45:12    FATAL - Running post_fatal callback...
04:45:12     INFO - Getting output from command: ['/builds/panda-0786/test/build/venv/bin/python', '-c', 'from distutils.sysconfig import get_python_lib; print(get_python_lib())']
04:45:12     INFO - Copy/paste: /builds/panda-0786/test/build/venv/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
04:45:12     INFO - Copy/paste: /builds/panda-0786/test/build/venv/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
04:45:12     INFO - Running post-action listener: _resource_record_post_action
04:45:12    FATAL - Uncaught exception: Traceback (most recent call last):
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/script.py", line 1111, in run
04:45:12    FATAL -     self.run_action(action)
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/script.py", line 1053, in run_action
04:45:12    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/script.py", line 994, in _possibly_run_method
04:45:12    FATAL -     return getattr(self, method_name)()
04:45:12    FATAL -   File "scripts/scripts/android_panda_talos.py", line 298, in download_and_extract
04:45:12    FATAL -     dirs['abs_talosdata_dir'])
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/mozilla/testing/testbase.py", line 202, in _download_unzip
04:45:12    FATAL -     error_level=FATAL)
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/script.py", line 235, in download_file
04:45:12    FATAL -     error_level=error_level,
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/script.py", line 483, in retry
04:45:12    FATAL -     self.log(error_message % {'action': action, 'attempts': n}, level=error_level)
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/log.py", line 55, in log
04:45:12    FATAL -     post_fatal_callback=self._post_fatal,
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/log.py", line 340, in log_message
04:45:12    FATAL -     post_fatal_callback(message=message, exit_code=exit_code)
04:45:12    FATAL -   File "scripts/scripts/android_panda_talos.py", line 388, in _post_fatal
04:45:12    FATAL -     self.close_request()
04:45:12    FATAL -   File "scripts/scripts/android_panda_talos.py", line 391, in close_request
04:45:12    FATAL -     mph = self.query_mozpool_handler(self.mozpool_device)
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/mozilla/testing/mozpool.py", line 46, in query_mozpool_handler
04:45:12    FATAL -     site_packages_path = self.query_python_site_packages_path()
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/python.py", line 121, in query_python_site_packages_path
04:45:12    FATAL -     'print(get_python_lib())'])
04:45:12    FATAL -   File "/builds/panda-0786/test/scripts/mozharness/base/script.py", line 764, in get_output_from_command
04:45:12    FATAL -     cwd=cwd, stderr=tmp_stderr, env=env)
04:45:12    FATAL -   File "/tools/python27/lib/python2.7/subprocess.py", line 679, in __init__
04:45:12    FATAL -     errread, errwrite)
04:45:12    FATAL -   File "/tools/python27/lib/python2.7/subprocess.py", line 1249, in _execute_child
04:45:12    FATAL -     raise child_exception
04:45:12    FATAL - OSError: [Errno 2] No such file or directory
04:45:12    FATAL - Running post_fatal callback...
}
In production :-)
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Hopefully should all be sorted now :-)
Severity: normal → blocker
dustin: before build.m.o was taken offline, your emails said the logs showed no-one was accessing it. How come this production load was missed? Can you turn build.m.o back on, asap, until this is resolved... and the loophole in log-analysis is fixed?
Flags: needinfo?(dustin)
(In reply to John O'Duinn [:joduinn] from comment #6)
> dustin: before build.m.o was taken offline, your emails said the logs showed
> no-one was accessing it. How come this production load was missed? Can you
> turn build.m.o back on, asap, until this is resolved... and the loophole in
> log-analysis is fixed?

Bah - firefox cache bug. I didnt know this bug was already fixed. Never mind with the reviving build.m.o. However, question about log-analysis stands... what else might be using build.m.o?
I switched to mozharness talos last Wednesday am, this is probably how it was missed in the logs.  Again, sorry for the breakage, I missed this in my scripts.
(In reply to Kim Moir [:kmoir] from comment #8)
> I switched to mozharness talos last Wednesday am, this is probably how it
> was missed in the logs.  Again, sorry for the breakage, I missed this in my
> scripts.

Ah, so kmoir's use was after dustin's last scan... hence the production snafu this morning. Got it.
Flags: needinfo?(dustin)
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.