1674736 - Can't run talos tabswitch on Windows - cannot import name Structure

Status: RESOLVED FIXED

(Testing :: Talos, defect, P2)

Description

[:Gijs (he/him)]

$ ./mach talos-test --suite tabswitch

<snip>

12:34:07     INFO - Calling ['objdir\\testing\\talos-venv\\Scripts\\python', 'mozilla-central\\testing\\talos\\talos\\run_tests.py', '--executablePath', 'objdir\\dist\\bin\\firefox', '--title', 'DESKTOP-MB3M8BI', '--develop', '--suite', 'tabswitch', '--log-tbpl-level=debug'] with output_timeout 3600
11:44:19    ERROR -  Traceback (most recent call last):
11:44:19     INFO -    File "/mozilla-central\testing\talos\talos\run_tests.py", line 24, in <module>
11:44:19     INFO -      from talos.ttest import TTest
11:44:19     INFO -    File "\mozilla-central\testing\talos\talos\ttest.py", line 29, in <module>
11:44:19     INFO -      from talos.cmanager import CounterManagement
11:44:19     INFO -    File "\mozilla-central\testing\talos\talos\cmanager.py", line 14, in <module>
11:44:19     INFO -      from talos.cmanager_win32 import WinCounterManager as DefaultCounterManager
11:44:19     INFO -    File "\mozilla-central\testing\talos\talos\cmanager_win32.py", line 10, in <module>
11:44:19     INFO -      from ctypes.wintypes import (
11:44:19     INFO -  ImportError: cannot import name 'Structure'
11:44:19    ERROR - Return code: 1

Comment 1

[:Gijs (he/him)]

This is on m-c 32e8e923c618 from this morning.

Comment 2

[Greg Mierzwinski [:sparky]]

:gijs, can you try to see if it still works if you put talos-test into this list? https://searchfox.org/mozilla-central/source/mach#14

I'm wondering if this is a regression from the py3 migration.

Comment 3

[Geoff Brown [:gbrown]]

It looks like Structure is normally imported directly from ctypes, rather than ctypes.wintypes:

https://searchfox.org/mozilla-central/rev/16d30bafd4e5276d6d3c632fb52a6c71e739cc44/third_party/python/virtualenv/virtualenv/util/subprocess/_win_subprocess.py#9

Comment 4

[:Gijs (he/him)]

(In reply to Greg Mierzwinski [:sparky] from comment #2)

:gijs, can you try to see if it still works if you put talos-test into this list? https://searchfox.org/mozilla-central/source/mach#14

Doesn't appear to help, I get the same error, though I didn't clobber anything in the meantime (do I need to?).

Comment 5

[Geoff Brown [:gbrown]]

Attachment: Bug 1674736 - [talos] Import Structure and pointer directly from ctypes; r=

Comment 6

[Geoff Brown [:gbrown]]

:gijs - Sorry, I don't have a convenient Windows environment. Can you try my patch?

Comment 7

[:Gijs (he/him)]

(In reply to Geoff Brown [:gbrown] from comment #6)

:gijs - Sorry, I don't have a convenient Windows environment. Can you try my patch?

The good news is it seems to fix that error. The bad news is it just moves it to:

23:37:10     INFO -  ImportError: cannot import name 'pointer'

from the same import statement.

I ran this bit of python on the python3 interpreter commandline in case it's helpful:

$ python3
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes.wintypes
>>> print(dir(ctypes.wintypes))
['ATOM', 'BOOL', 'BOOLEAN', 'BYTE', 'CHAR', 'COLORREF', 'DOUBLE', 'DWORD', 'FILETIME', 'FLOAT', 'HACCEL', 'HANDLE', 'HBITMAP', 'HBRUSH', 'HCOLORSPACE', 'HDC', 'HDESK', 'HDWP', 'HENHMETAFILE', 'HFONT', 'HGDIOBJ', 'HGLOBAL', 'HHOOK', 'HICON', 'HINSTANCE', 'HKEY', 'HKL', 'HLOCAL', 'HMENU', 'HMETAFILE', 'HMODULE', 'HMONITOR', 'HPALETTE', 'HPEN', 'HRGN', 'HRSRC', 'HSTR', 'HTASK', 'HWINSTA', 'HWND', 'INT', 'LANGID', 'LARGE_INTEGER', 'LCID', 'LCTYPE', 'LGRPID', 'LONG', 'LPARAM', 'LPBOOL', 'LPBYTE', 'LPCOLESTR', 'LPCOLORREF', 'LPCSTR', 'LPCVOID', 'LPCWSTR', 'LPDWORD', 'LPFILETIME', 'LPHANDLE', 'LPHKL', 'LPINT', 'LPLONG', 'LPMSG', 'LPOLESTR', 'LPPOINT', 'LPRECT', 'LPRECTL', 'LPSC_HANDLE', 'LPSIZE', 'LPSIZEL', 'LPSTR', 'LPUINT', 'LPVOID', 'LPWIN32_FIND_DATAA', 'LPWIN32_FIND_DATAW', 'LPWORD', 'LPWSTR', 'MAX_PATH', 'MSG', 'OLESTR', 'PBOOL', 'PBOOLEAN', 'PBYTE', 'PCHAR', 'PDWORD', 'PFILETIME', 'PFLOAT', 'PHANDLE', 'PHKEY', 'PINT', 'PLARGE_INTEGER', 'PLCID', 'PLONG', 'PMSG', 'POINT', 'POINTL', 'PPOINT', 'PPOINTL', 'PRECT', 'PRECTL', 'PSHORT', 'PSIZE', 'PSIZEL', 'PSMALL_RECT', 'PUINT', 'PULARGE_INTEGER', 'PULONG', 'PUSHORT', 'PWCHAR', 'PWIN32_FIND_DATAA', 'PWIN32_FIND_DATAW', 'PWORD', 'RECT', 'RECTL', 'RGB', 'SC_HANDLE', 'SERVICE_STATUS_HANDLE', 'SHORT', 'SIZE', 'SIZEL', 'SMALL_RECT', 'UINT', 'ULARGE_INTEGER', 'ULONG', 'USHORT', 'VARIANT_BOOL', 'WCHAR', 'WIN32_FIND_DATAA', 'WIN32_FIND_DATAW', 'WORD', 'WPARAM', '_COORD', '_FILETIME', '_LARGE_INTEGER', '_POINTL', '_RECTL', '_SMALL_RECT', '_ULARGE_INTEGER', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'ctypes', 'tagMSG', 'tagPOINT', 'tagRECT', 'tagSIZE']

It looks like pointer does exist on ctypes, like Structure.

I can also confirm that if I run python2, the same prints do list Structure and pointer as existing in ctypes.wintypes.

I don't know how to reconcile that with things still not working if I run the same thing under python2. I did just try that again, and it still fails with the pointer error - from reading

23:44:45     INFO - Calling [u'objdir\\testing\\talos-venv\\Scripts\\python', u'mozilla-central\\testing\\talos\\talos\\run_tests.py', '--executablePath', u'objdir\\dist\\bin\\firefox', '--title', u'DESKTOP-MB3M8BI', u'--develop', u'--suite', u'tabswitch', '--log-tbpl-level=debug'] with output_timeout 3600

I guess this is because it uses just "python" and in the venv, that points to python3 now? Maybe I'd have to rebuild said venv, but I don't know how to...

Comment 8

[Geoff Brown [:gbrown]]

(In reply to :Gijs (he/him) from comment #7)

I guess this is because it uses just "python" and in the venv, that points to python3 now? Maybe I'd have to rebuild said venv, but I don't know how to...

I think you can just delete objdir\testing\talos-venv, and the next run of 'mach talos-test' will re-build it.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0a35604d5a46
[talos] Import Structure and pointer directly from ctypes; r=perftest-reviewers,Gijs,sparky

Comment 10

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/0a35604d5a46