Closed Bug 2070810 Opened 5 days ago Closed 4 days ago

{standard input}:17678: Error: no such instruction: `{vex} vpdpbusd (%r8),%ymm0,%ymm5'

Categories

(Firefox Build System :: Toolchains, defect)

Product:
Firefox Build System
Component:
Toolchains
Type:
defect

Tracking

(firefox158 fixed)

Status:
RESOLVED FIXED
Milestone:
158 Branch
Tracking Flags:
Tracking Status
firefox158 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(2 files)

Bug 2070810 - Upgrade rust toolchain used to build toolchains to 1.91.
48 bytes, text/x-phabricator-request
Details | Review
Bug 2070810 - Upgrade sccache to current main.
48 bytes, text/x-phabricator-request
Details | Review

Landing bug 2069184 led to this error in base-toolchains tasks:

[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}: Assembler messages:
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:17678: Error: no such instruction: `{vex} vpdpbusd (%r8),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:17792: Error: no such instruction: `{vex} vpdpbusd 32(%r8),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:17952: Error: no such instruction: `{vex} vpdpbusd -192(%rsi),%ymm0,%ymm4'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18101: Error: no such instruction: `{vex} vpdpbusd -160(%rsi),%ymm0,%ymm7'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18134: Error: no such instruction: `{vex} vpdpbusd -128(%rsi),%ymm0,%ymm2'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18145: Error: no such instruction: `{vex} vpdpbusd -96(%rsi),%ymm0,%ymm6'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18156: Error: no such instruction: `{vex} vpdpbusd -64(%rsi),%ymm0,%ymm3'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18167: Error: no such instruction: `{vex} vpdpbusd -32(%rsi),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18702: Error: no such instruction: `{vex} vpdpbusd (%rsi),%ymm0,%ymm9'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18785: Error: no such instruction: `{vex} vpdpbusd 32(%rsi),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18862: Error: no such instruction: `{vex} vpdpbusd (%rax),%ymm0,%ymm9'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18921: Error: no such instruction: `{vex} vpdpbusd 32(%rax),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:18991: Error: no such instruction: `{vex} vpdpbusd 64(%rax),%ymm0,%ymm4'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:19101: Error: no such instruction: `{vex} vpdpbusd 96(%rax),%ymm0,%ymm7'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:19132: Error: no such instruction: `{vex} vpdpbusd -128(%rax),%ymm0,%ymm2'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:19141: Error: no such instruction: `{vex} vpdpbusd -96(%rax),%ymm0,%ymm6'
[task 2026-09-09T20:11:46.904+00:00] 20:11:46     INFO -  {standard input}:19150: Error: no such instruction: `{vex} vpdpbusd -64(%rax),%ymm0,%ymm3'
[task 2026-09-09T20:11:46.905+00:00] 20:11:46     INFO -  {standard input}:19159: Error: no such instruction: `{vex} vpdpbusd -32(%rax),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.905+00:00] 20:11:46     INFO -  {standard input}:20187: Error: no such instruction: `{vex} vpdpbusd (%rax),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.905+00:00] 20:11:46     INFO -  {standard input}:20279: Error: no such instruction: `{vex} vpdpbusd 32(%rax),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.905+00:00] 20:11:46     INFO -  {standard input}:20573: Error: no such instruction: `{vex} vpdpbusd (%rdi),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:20686: Error: no such instruction: `{vex} vpdpbusd 32(%rdi),%ymm0,%ymm8'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:20885: Error: no such instruction: `{vex} vpdpbusd -192(%rsi),%ymm0,%ymm4'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21034: Error: no such instruction: `{vex} vpdpbusd -160(%rsi),%ymm0,%ymm7'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21067: Error: no such instruction: `{vex} vpdpbusd -128(%rsi),%ymm0,%ymm2'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21078: Error: no such instruction: `{vex} vpdpbusd -96(%rsi),%ymm0,%ymm6'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21089: Error: no such instruction: `{vex} vpdpbusd -64(%rsi),%ymm0,%ymm3'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21100: Error: no such instruction: `{vex} vpdpbusd -32(%rsi),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21635: Error: no such instruction: `{vex} vpdpbusd (%rsi),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21718: Error: no such instruction: `{vex} vpdpbusd 32(%rsi),%ymm0,%ymm8'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21795: Error: no such instruction: `{vex} vpdpbusd (%rax),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21854: Error: no such instruction: `{vex} vpdpbusd 32(%rax),%ymm0,%ymm8'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:21924: Error: no such instruction: `{vex} vpdpbusd 64(%rax),%ymm0,%ymm4'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22034: Error: no such instruction: `{vex} vpdpbusd 96(%rax),%ymm0,%ymm7'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22065: Error: no such instruction: `{vex} vpdpbusd -128(%rax),%ymm0,%ymm2'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22074: Error: no such instruction: `{vex} vpdpbusd -96(%rax),%ymm0,%ymm6'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22083: Error: no such instruction: `{vex} vpdpbusd -64(%rax),%ymm0,%ymm3'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22092: Error: no such instruction: `{vex} vpdpbusd -32(%rax),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22978: Error: no such instruction: `{vex} vpdpbusd 32(%rax),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:22989: Error: no such instruction: `{vex} vpdpbusd (%rax),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.908+00:00] 20:11:46     INFO -  {standard input}:23402: Error: no such instruction: `{vex} vpdpbusd (%rcx),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23413: Error: no such instruction: `{vex} vpdpbusd 32(%rcx),%ymm0,%ymm8'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23424: Error: no such instruction: `{vex} vpdpbusd 64(%rcx),%ymm0,%ymm4'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23435: Error: no such instruction: `{vex} vpdpbusd 96(%rcx),%ymm0,%ymm7'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23446: Error: no such instruction: `{vex} vpdpbusd 128(%rcx),%ymm0,%ymm2'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23457: Error: no such instruction: `{vex} vpdpbusd 160(%rcx),%ymm0,%ymm6'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23468: Error: no such instruction: `{vex} vpdpbusd 192(%rcx),%ymm0,%ymm3'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:23564: Error: no such instruction: `{vex} vpdpbusd 224(%rcx),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24150: Error: no such instruction: `{vex} vpdpbusd -256(%rax),%ymm0,%ymm1'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24159: Error: no such instruction: `{vex} vpdpbusd -224(%rax),%ymm0,%ymm8'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24168: Error: no such instruction: `{vex} vpdpbusd -192(%rax),%ymm0,%ymm4'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24177: Error: no such instruction: `{vex} vpdpbusd -160(%rax),%ymm0,%ymm7'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24186: Error: no such instruction: `{vex} vpdpbusd -128(%rax),%ymm0,%ymm2'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24195: Error: no such instruction: `{vex} vpdpbusd -96(%rax),%ymm0,%ymm6'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24204: Error: no such instruction: `{vex} vpdpbusd -64(%rax),%ymm0,%ymm3'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46     INFO -  {standard input}:24259: Error: no such instruction: `{vex} vpdpbusd -32(%rax),%ymm0,%ymm5'
[task 2026-09-09T20:11:46.912+00:00] 20:11:46    ERROR -  gmake[4]: *** [/builds/worker/checkouts/gecko/config/rules.mk:656: GemmologyEngineAVXVNNI.o] Error 1

What happened?

The first difference is this in the configure log before and after landing 2069184.
Before:

checking for -mavxvnni support... no

After:

checking for -mavxvnni support... yes

This doesn't really tell why. Getting config.log (after bug 2070759) from shows the difference:
Before:

INFO: checking for -mavxvnni support... 
DEBUG: Creating `/tmp/conftest1fqyb4id.cpp` with content:
DEBUG: | #include <immintrin.h>
DEBUG: |       int
DEBUG: |       main(void)
DEBUG: |       {
DEBUG: | 
DEBUG: | __m256i a = _mm256_set1_epi32(1);
DEBUG: | return _mm256_extract_epi32(_mm256_dpbusd_avx_epi32(a, a, a), 0);
DEBUG: | 
DEBUG: |         ;
DEBUG: |         return 0;
DEBUG: |       }
DEBUG: Executing: `/builds/worker/fetches/sccache/sccache /builds/worker/fetches/gcc/bin/g++ --sysroot /builds/worker/fetches/sysroot-x86_64-linux-gnu -std=gnu++20 -U__cplusplus -D__cplusplus=202002L -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/c++/10 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu/c++/10 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include /tmp/conftest1fqyb4id.cpp -mavxvnni -c`
DEBUG: The command returned non-zero exit status 1.
DEBUG: Its error output was:
DEBUG: | g++: error: unrecognized command-line option '-mavxvnni'; did you mean '-mavx512vnni'?
INFO: no

After:

INFO: checking for -mavxvnni support... 
DEBUG: Creating `/tmp/conftestb2zfhh3o.cpp` with content:
DEBUG: | #include <immintrin.h>
DEBUG: |       int
DEBUG: |       main(void)
DEBUG: |       {
DEBUG: | 
DEBUG: | __m256i a = _mm256_set1_epi32(1);
DEBUG: | return _mm256_extract_epi32(_mm256_dpbusd_avx_epi32(a, a, a), 0);
DEBUG: | 
DEBUG: |         ;
DEBUG: |         return 0;
DEBUG: |       }
DEBUG: Executing: `/builds/worker/fetches/sccache/sccache /builds/worker/fetches/gcc/bin/g++ --sysroot /builds/worker/fetches/sysroot-x86_64-linux-gnu -std=gnu++20 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/c++/11 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu/c++/11 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include /tmp/conftestb2zfhh3o.cpp -mavxvnni -c`
INFO: yes

Now, here's the kicker: running that last "Executing" command locally does fail.
So for some reason, sccache gets a cached result for this compile that is a success, while disabling sccache, using sccache with an empty cache, or using SCCACHE_RECACHE leads to the expected failure.

The proper config.log output looks like:

DEBUG: Creating `/tmp/conftestawikm8_j.cpp` with content:
DEBUG: | #include <immintrin.h>
DEBUG: |       int
DEBUG: |       main(void)
DEBUG: |       {
DEBUG: | 
DEBUG: | __m256i a = _mm256_set1_epi32(1);
DEBUG: | return _mm256_extract_epi32(_mm256_dpbusd_avx_epi32(a, a, a), 0);
DEBUG: | 
DEBUG: |         ;
DEBUG: |         return 0;
DEBUG: |       }
DEBUG: Executing: `/builds/worker/fetches/sccache/sccache /builds/worker/fetches/gcc/bin/g++ --sysroot /builds/worker/fetches/sysroot-x86_64-linux-gnu -std=gnu++20 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/c++/11 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu/c++/11 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include /tmp/conftestawikm8_j.cpp -mavxvnni -c`
DEBUG: The command returned non-zero exit status 1.
DEBUG: Its error output was:
DEBUG: | /tmp/ccyurFKr.s: Assembler messages:
DEBUG: | /tmp/ccyurFKr.s:27: Error: no such instruction: `{vex} vpdpbusd %ymm2,%ymm1,%ymm0'
INFO: no

What's going on here, is that another build, namely the linux64-gcc/opt build, runs the same command during configure, and we're getting the cache from /that/ run. But why? Well, because GCC doesn't work alone. It relies on GAS. GAS is part of binutils. And the linux64-gcc/opt and build-linux64-base-toolchains/opt are using a different version of binutils. The discrepancy in binutils version is the reason we added code to those flag tests in the first place! (bug 2069190).

So whichever build running these configure tests comes first on a cache miss wins and decides the fate of the other builds, whether binutils supports the instruction sets the flag enables or not. And when it's a task using the newer version of binutils that wins the race, a task using the old version of binutils gets a positive flag check in configure, and the build fails because that result doesn't reflect reality.

This includes, among other things, a fix for cache collisions from using
GCC with different versions of binutils.

https://hg.mozilla.org/mozilla-central/rev/39e4e84afdcd
https://hg.mozilla.org/mozilla-central/rev/9e7e3b9f63c9

Status: NEW → RESOLVED
Closed: 4 days ago
status-firefox158: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 158 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: