868695 - Getting "struct.error: unpack_from requires a buffer of at least 32 bytes" when idl has errors, misleading error message

Status: NEW

(Firefox Build System :: General, defect)

Description

[Honza Bambas (:mayhemer)]

Attachment: local patch causing the error

I've added few new .idl files to a new xpt module.  When building the directory (netwerk/cache2) I'm getting this error (gum project branch + the patch):

$ mach -v build netwerk/cache2
 0:00.28 c:/Mozilla/mozilla-build/msys/bin/sh.exe -c c:/Mozilla/src/gum/build/pymake/make.py -C netwerk/cache2 -j8 -s
 0:00.28 Environment: {'LIB': 'C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v7.0A\\\\lib;c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\\\Lib
;;c:\\Mozilla\\mozilla-build\\atlthunk_compat', 'VS100COMNTOOLS': 'c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\Common7\\Tools\\', 'SYSTEMROOT': 'C:
\\Windows', 'SHELL': 'c:/Mozilla/mozilla-build/msys/bin/sh', 'WINCURVERKEY': 'HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion', 'MOZ_MAXWINSDK': '999999',
'WIX_351728_PATH': 'c:/Mozilla/mozilla-build/wix-351728', 'SDK6KEY': 'HKLM\\SOFTWARE\\Microsoft\\Microsoft SDKs\\Windows\\v6.0', 'MSVCROOTKEY': 'HKLM\\SOFTWAR
E\\Wow6432Node\\Microsoft\\VisualStudio', 'COMSPEC': 'C:\\Windows\\system32\\cmd.exe', 'MSVC8KEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VisualStudio\\8.0\
\Setup\\VC', 'MSVC71KEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VisualStudio\\7.1\\Setup\\VC', 'MSVC11KEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\Visual
Studio\\11.0\\Setup\\VC', 'USEPSDKATL': '1', 'HOMEDRIVE': 'C:', 'SYSTEMDRIVE': 'C:', 'PSDKVER': '4', 'VC9EXPRESSDIR': 'C:\\Program Files (x86)\\Microsoft Visu
al Studio 9.0\\VC\\', 'PROCESSOR_LEVEL': '6', 'MACHTYPE': 'i686-pc-msys', 'OS': 'Windows_NT', 'SDK2003SP1KEY': 'HKLM\\SOFTWARE\\Microsoft\\MicrosoftSDK\\Insta
lledSDKs\\8F9E5EF3-A9A5-491B-A889-C58EFFECE8B3', 'MSVC11EXPRESSKEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VCExpress\\11.0\\Setup\\VC', 'MSVC10EXPRESSKEY':
 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VCExpress\\10.0\\Setup\\VC', 'MSVC9EXPRESSKEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VCExpress\\9.0\\Setup\\VC',
 'CVS_RSH': 'ssh', 'PSDKDIR': 'C:\\Program Files\\Microsoft Platform SDK\\', 'INPUTRC': 'c:/Mozilla/mozilla-build/msys/etc/inputrc', 'SDK7KEY': 'HKLM\\SOFTWAR
E\\Microsoft\\Microsoft SDKs\\Windows\\v7.0', 'PS1': '\\[\\033]0;$MSYSTEM:\\w\\007\n\\033[32m\\]\\u@\\h \\[\\033[33m\\w\\033[0m\\]\n$ ', 'AMDAPPSDKROOT': 'C:\
\Program Files (x86)\\AMD APP\\', 'MSYSTEM': 'MINGW32', '___NSPR_LOG_FILE': 'c:\\Mozilla\\regular-account.log', 'TEMP': 'C:/Users/mayhemer/AppData/Local/Temp'
, '!EXITCODE': '00000000', 'SHLVL': '1', 'COMMONPROGRAMFILES(X86)': 'C:\\Program Files (x86)\\Common Files', 'APR_ICONV_PATH': 'c:/Mozilla/mozilla-build/svn-w
in32-1.6.3/iconv', 'SDK80KEY': 'HKLM\\SOFTWARE\\Microsoft\\Windows Kits\\Installed Roots', 'EDITOR': 'emacs.exe', 'HOMEPATH': '\\', 'VS80COMNTOOLS': 'C:\\Prog
ram Files (x86)\\Microsoft Visual Studio 8\\Common7\\Tools\\', 'WINDOWSSDKDIR': 'C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v7.0A\\', 'MOZBUILDDIR': 'c
:\\Mozilla\\mozilla-build\\', 'MOZILLABUILD': 'c:\\Mozilla\\mozilla-build\\', 'LOGONSERVER': '\\\\I7', 'SESSIONNAME': 'Console', 'VC8EXPRESSDIR': 'C:\\Program
 Files (x86)\\Microsoft Visual Studio 8\\VC\\', '_': './mach', 'OPENSSL_CONF': 'C:\\Program Files\\OpenSSL-Win64\\bin\\openssl.cfg', 'SDK7AKEY': 'HKLM\\SOFTWA
RE\\Microsoft\\Microsoft SDKs\\Windows\\v7.0A', 'APPDATA': 'C:\\Users\\mayhemer\\AppData\\Roaming', 'SDKVER': '7', 'VBOX_INSTALL_PATH': 'C:\\Program Files\\Or
acle\\VirtualBox\\', 'SDK2003SP2KEY': 'HKLM\\SOFTWARE\\Microsoft\\MicrosoftSDK\\InstalledSDKs\\D2FF9F89-8AA2-4373-8A31-C838BF4DBBE1', 'MSVC10KEY': 'HKLM\\SOFT
WARE\\Wow6432Node\\Microsoft\\VisualStudio\\10.0\\Setup\\VC', 'SDKROOTKEY': 'HKLM\\SOFTWARE\\Microsoft\\MicrosoftSDK\\InstalledSDKs', 'FRAMEWORKVERSION': 'v4.
0.30319', 'TMP': 'C:/Users/mayhemer/AppData/Local/Temp', 'COMPUTERNAME': 'I7', 'HISTFILE': 'c:/Mozilla/mozilla-build/msys/home/mayhemer/.bash_history', 'USERD
OMAIN': 'i7', 'MOZCONFIG': 'c:/Mozilla/confs/bd', 'VS90COMNTOOLS': 'C:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\Common7\\Tools\\', 'COMMONPROGRAMFIL
ES': 'C:\\Program Files (x86)\\Common Files', 'HOME': 'c:/Users/mayhemer', 'SDKDIR': 'C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v7.0A\\', 'INCLUDE': '
C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v7.0A\\\\include;C:\\Program Files\\Microsoft Platform SDK\\\\include\\atl;c:\\Program Files (x86)\\Microsof
t Visual Studio 10.0\\VC\\\\Include;c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\\\Include\\Sys;', 'PROCESSOR_ARCHITECTURE': 'x86', 'ALLUSERSPRO
FILE': 'C:\\ProgramData', 'OSTYPE': 'msys', 'FRAMEWORKDIR': 'C:\\Windows\\Microsoft.NET\\Framework\\', 'PROGRAMW6432': 'C:\\Program Files', 'USERNAME': 'mayhe
mer', 'MSVC9KEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VisualStudio\\9.0\\Setup\\VC', 'PROMPT': '$P$G', 'MOZ_TOOLS': 'c:\\Mozilla\\mozilla-build\\moztools
', 'COMMANDER_DRIVE': 'C:', 'COMMANDER_EXE': 'C:\\Program Files\\Total Commander\\TOTALCMD64.EXE', 'HOSTTYPE': 'i686', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.V
BE;.JS;.JSE;.WSF;.WSH;.MSC', 'WINDIR': 'C:\\Windows', 'MSVCEXPROOTKEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VCExpress', 'NUMBER_OF_PROCESSORS': '8', 'DEV
ENVDIR': 'c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\Common7\\IDE\\', 'PROCESSOR_ARCHITEW6432': 'AMD64', 'PUBLIC': 'C:\\Users\\Public', 'USERPROFI
LE': 'C:\\Users\\mayhemer', 'PSMODULEPATH': 'C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\', 'FRAMEWORKVERSION32': 'v4.0.30319', 'TEMPVC9DIR': 'C:
\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\', 'SDKMINORVER': '0A', 'COMMANDER_PATH': 'C:\\Program Files\\Total Commander', 'LIBPATH': 'C:\\Window
s\\Microsoft.NET\\Framework\\v4.0.30319;C:\\Windows\\Microsoft.NET\\Framework\\v3.5;c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\LIB;', 'PROCESS
OR_IDENTIFIER': 'Intel64 Family 6 Model 26 Stepping 5, GenuineIntel', 'LOGNAME': 'mayhemer', 'PROGRAMFILES': 'C:\\Program Files (x86)', 'PROCESSOR_REVISION':
'1a05', 'PATH': 'c:\\Mozilla\\mozilla-build\\msys\\local\\bin;c:\\Mozilla\\mozilla-build\\wget;c:\\Mozilla\\mozilla-build\\7zip;c:\\Mozilla\\mozilla-build\\bl
at261\\full;c:\\Mozilla\\mozilla-build\\python;c:\\Mozilla\\mozilla-build\\svn-win32-1.6.3\\bin;c:\\Mozilla\\mozilla-build\\upx203w;c:\\Mozilla\\mozilla-build
\\emacs-22.3\\bin;c:\\Mozilla\\mozilla-build\\info-zip;c:\\Mozilla\\mozilla-build\\nsis-2.22;c:\\Mozilla\\mozilla-build\\nsis-2.33u;c:\\Mozilla\\mozilla-build
\\nsis-2.46u;c:\\Mozilla\\mozilla-build\\wix-351728;c:\\Mozilla\\mozilla-build\\hg;c:\\Mozilla\\mozilla-build\\python\\Scripts;c:\\Mozilla\\mozilla-build\\kdi
ff3;c:\\Mozilla\\mozilla-build\\yasm;.;c:\\Mozilla\\mozilla-build\\msys\\local\\bin;c:\\Mozilla\\mozilla-build\\msys\\mingw\\bin;c:\\Mozilla\\mozilla-build\\m
sys\\bin;c:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v7.0A\\bin;c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\Bin;c:\\Program Files (x86)\\
Microsoft Visual Studio 10.0\\Common7\\IDE;c:\\Windows\\System32;c:\\Windows;c:\\Windows\\System32\\Wbem;c:\\Mozilla\\mozilla-build\\moztools\\bin;c:\\Mozilla
\\mozilla-build\\vim\\vim72;c:\\Program Files (x86)\\Git\\bin', 'PROGRAMFILES(X86)': 'C:\\Program Files (x86)', 'SSH_AGENT_PID': '3308', 'MAKE_MODE': 'unix',
'TERM': 'cygwin', 'WIN64': '1', 'VSINSTALLDIR': 'c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\', '!::': '::\\', 'LOCALAPPDATA': 'C:\\Users\\mayhemer
\\AppData\\Local', 'SDK61KEY': 'HKLM\\SOFTWARE\\Microsoft\\Microsoft SDKs\\Windows\\v6.1', 'MSVC8EXPRESSKEY': 'HKLM\\SOFTWARE\\Wow6432Node\\Microsoft\\VCExpre
ss\\8.0\\Setup\\VC', 'PROGRAMDATA': 'C:\\ProgramData', 'SSH_AUTH_SOCK': 'C:/Users/mayhemer/AppData/Local/Temp/ssh-NaBigX4784/agent.4784', 'SDK6AKEY': 'HKLM\\S
OFTWARE\\Microsoft\\Microsoft SDKs\\Windows\\v6.0A', '___NSPR_LOG_MODULES': 'timestamp,negotiateauth:5,nsHttp:5', 'SDK71KEY': 'HKLM\\SOFTWARE\\Microsoft\\Micr
osoft SDKs\\Windows\\v7.1', 'VC10EXPRESSDIR': 'c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\', 'TEMPVC10DIR': 'c:\\Program Files (x86)\\Microsof
t Visual Studio 10.0\\VC\\', 'FP_NO_HOST_CHECK': 'NO', 'FRAMEWORK35VERSION': 'v3.5', 'MOZ_MSVCVERSION': '10', 'COMMANDER_INI': 'C:\\Users\\mayhemer\\AppData\\
Roaming\\GHISLER\\wincmd.ini', 'OLDPWD': 'c:/Mozilla/src/mozilla-central', 'DXSDK_DIR': 'C:\\Program Files (x86)\\Microsoft DirectX SDK (June 2010)\\', 'PWD':
 'c:/Mozilla/src/gum', 'FRAMEWORKDIR32': 'C:\\Windows\\Microsoft.NET\\Framework\\', 'COMMONPROGRAMW6432': 'C:\\Program Files\\Common Files', 'VCINSTALLDIR': '
c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\'}
 0:01.20 Traceback (most recent call last):
 0:01.20   File "c:/Mozilla/src/gum/_obj-browser-debug/dist/sdk/bin/xpt.py", line 1381, in <module>
 0:01.21     xpt_link(sys.argv[3:]).write(sys.argv[2])
 0:01.21   File "c:/Mozilla/src/gum/_obj-browser-debug/dist/sdk/bin/xpt.py", line 1275, in xpt_link
 0:01.21     t = read_input(f)
 0:01.21   File "c:/Mozilla/src/gum/_obj-browser-debug/dist/sdk/bin/xpt.py", line 1263, in read_input
 0:01.21     return Typelib.read(i)
 0:01.21   File "c:/Mozilla/src/gum/_obj-browser-debug/dist/sdk/bin/xpt.py", line 1083, in read
 0:01.21     data_pool_offset) = Typelib._header.unpack_from(data)
 0:01.21 struct.error: unpack_from requires a buffer of at least 32 bytes
 0:01.21 c:\Mozilla\src\gum\config\rules.mk:1375:0: command 'c:/Mozilla/src/gum/_obj-browser-debug/_virtualenv/Scripts/python.exe c:/Mozilla/src/gum/_obj-brow
ser-debug/dist/sdk/bin/xpt.py link _xpidlgen/necko_cache2.xpt _xpidlgen/nsICacheEntry.xpt _xpidlgen/nsICacheEntryOpenCallback.xpt _xpidlgen/nsICacheStorage.xp
t _xpidlgen/nsICacheStorageService.xpt' failed, return code 1
 0:01.22 <../../dist/bin/components/necko_cache2.xpt>: Found error
 0:01.22 <libs>: Found error
 0:01.29 c:\Mozilla\src\gum\config\rules.mk:633:0: command 'c:/Mozilla/mozilla-build/python/python.exe c:/Mozilla/src/gum/build/pymake/pymake/../make.py libs'
 failed, return code 2



Not sure I am doing something wrong or this is some build system regression.  Pymaking -C the dir produces the same error.  Full build as well.  .idls -> .h works well and my new .cpps compile.

Comment 1

[Honza Bambas (:mayhemer)]

Note: having "XPIDL_SOURCES += [" in moz.build (the added '+') gives the same result.

Comment 2

[Honza Bambas (:mayhemer)]

So... the problem was that nsICacheEntry.idl contained errors.  But I would expect to get information about the idl file being corrupted and not this cryptic error that doesn't lead to finding the true problem.

I had to overlook the error during full build (lost in parallelism) and then I was getting only the struct.error message.

I'm leaving this open, updating the title.

Comment 3

[Gregory Szorc [:gps]]

Dupe of bug 809978?

Comment 4

[Honza Bambas (:mayhemer)]

(In reply to Gregory Szorc [:gps] from comment #3)
> Dupe of bug 809978?

Hard to say for me.