1485946 - Permafailure z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2057: expected constant expression

Status: RESOLVED FIXED

(Toolkit :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: toros [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=195696856&repo=mozilla-central

https://queue.taskcluster.net/v1/task/cauHFyuHQymcE4CyzOBSyg/runs/0/artifacts/public/logs/live_backing.log

10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/media/libvpx'
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/media/libvpx'
10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/media/libvpx'
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/media/libaom'
10:01:41     INFO -  z:/build/build/src/obj-firefox/_virtualenvs/init/Scripts/python.exe -m mozbuild.action.cl  z:/build/build/src/vs2017_15.6.6/VC/bin/Hostx64/x64/cl.exe -Foaom_codec.obj -c  -DDEBUG=1 -DSTATIC_EXPORTABLE_JS_API -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -Iz:/build/build/src/media/libaom -Iz:/build/build/src/obj-firefox/media/libaom -Iz:/build/build/src/media/libaom/config/win/x64 -Iz:/build/build/src/media/libaom/config -Iz:/build/build/src/third_party/aom -Iz:/build/build/src/obj-firefox/dist/include -Iz:/build/build/src/obj-firefox/dist/include/nspr -Iz:/build/build/src/obj-firefox/dist/include/nss -MD -FI z:/build/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -utf-8 -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -Gw -wd4244 -wd4267 -we4553 -Zi -O1 -Oi -Oy-    -Fdgenerated.pdb -FS  z:/build/build/src/third_party/aom/aom/src/aom_codec.c
10:01:41     INFO -  aom_codec.c
10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/media/libaom'
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/media/libaom'
10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/media/libaom'
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/third_party/prio'
10:01:41     INFO -  z:/build/build/src/obj-firefox/_virtualenvs/init/Scripts/python.exe -m mozbuild.action.cl  z:/build/build/src/vs2017_15.6.6/VC/bin/Hostx64/x64/cl.exe -Foencrypt.obj -c  -DDEBUG=1 -DPRIO_BUILD_LIBRARY -DSTATIC_EXPORTABLE_JS_API -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -Iz:/build/build/src/third_party/prio -Iz:/build/build/src/obj-firefox/third_party/prio -Iz:/build/build/src/security/nss/lib/freebl/mpi -Iz:/build/build/src/third_party/msgpack/include -Iz:/build/build/src/third_party/prio/include -Iz:/build/build/src/obj-firefox/dist/include -Iz:/build/build/src/obj-firefox/dist/include/nspr -Iz:/build/build/src/obj-firefox/dist/include/nss -MD -FI z:/build/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -utf-8 -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -Gw -wd4244 -wd4267 -we4553 -Zi -O1 -Oi -Oy-    -Fdgenerated.pdb -FS  z:/build/build/src/third_party/prio/prio/encrypt.c
10:01:41     INFO -  encrypt.c
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2057: expected constant expression
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2466: cannot allocate an array of constant size 0
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2133: 'key_bytes': unknown size
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(84): error C2057: expected constant expression
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(84): error C2466: cannot allocate an array of constant size 0
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(84): error C2133: 'spki_data': unknown size
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(281): error C2057: expected constant expression
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(281): error C2466: cannot allocate an array of constant size 0
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(281): error C2133: 'aadBuf': unknown size
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(319): error C2057: expected constant expression
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(319): error C2466: cannot allocate an array of constant size 0
10:01:41     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(319): error C2133: 'aad_buf': unknown size
10:01:41     INFO -  z:/build/build/src/config/rules.mk:779: recipe for target 'encrypt.obj' failed
10:01:41     INFO -  mozmake.EXE[4]: *** [encrypt.obj] Error 2
10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/third_party/prio'
10:01:41     INFO -  z:/build/build/src/config/recurse.mk:74: recipe for target 'third_party/prio/target' failed
10:01:41     INFO -  mozmake.EXE[3]: *** [third_party/prio/target] Error 2
10:01:41     INFO -  mozmake.EXE[3]: *** Waiting for unfinished jobs....
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/config/external/icu/i18n'
10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/config/external/icu/i18n'
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/config/external/icu/i18n'
10:01:41     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/config/external/icu/i18n'
10:01:41     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/media/libvorbis'
10:01:41     INFO -  z:/build/build/src/obj-firefox/_virtualenvs/init/Scripts/python.exe -m mozbuild.action.cl  z:/build/build/src/vs2017_15.6.6/VC/bin/Hostx64/x64/cl.exe -FoUnified_c_media_libvorbis0.obj -c  -DDEBUG=1 -DSTATIC_EXPORTABLE_JS_API -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -Iz:/build/build/src/media/libvorbis -Iz:/build/build/src/obj-firefox/media/libvorbis -Iz:/build/build/src/media/libvorbis/lib -Iz:/build/build/src/obj-firefox/dist/include -Iz:/build/build/src/obj-firefox/dist/include/nspr -Iz:/build/build/src/obj-firefox/dist/include/nss -MD -FI z:/build/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -utf-8 -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -Gw -wd4244 -wd4267 -we4553 -Zi -O1 -Oi -Oy-    -Fdgenerated.pdb -FS  z:/build/build/src/obj-firefox/media/libvorbis/Unified_c_media_libvorbis0.c
10:01:41     INFO -  Unified_c_media_libvorbis0.c

Comment 1

[Intermittent Failures Robot]

24 failures in 478 pushes (0.05 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-central: 24

Platform breakdown:
* windows2012-32: 12
* windows2012-64: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1485946&startday=2018-08-24&endday=2018-08-24&tree=all

Comment 2

[Natalia Csoregi [:nataliaCs]]

There have been 48 failures within the last 7 days, on windows 2012 x32/x64.

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=195945163&repo=mozilla-central&lineNumber=6031

22:28:34     INFO -  z:/build/build/src/obj-firefox/_virtualenvs/init/Scripts/python.exe -m mozbuild.action.cl  z:/build/build/src/vs2017_15.6.6/VC/bin/Hostx64/x86/cl.exe -Foencrypt.obj -c  -DDEBUG=1 -DPRIO_BUILD_LIBRARY -DSTATIC_EXPORTABLE_JS_API -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -Iz:/build/build/src/third_party/prio -Iz:/build/build/src/obj-firefox/third_party/prio -Iz:/build/build/src/security/nss/lib/freebl/mpi -Iz:/build/build/src/third_party/msgpack/include -Iz:/build/build/src/third_party/prio/include -Iz:/build/build/src/obj-firefox/dist/include -Iz:/build/build/src/obj-firefox/dist/include/nspr -Iz:/build/build/src/obj-firefox/dist/include/nss -MD -FI z:/build/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -utf-8 -nologo -wd4091 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -arch:SSE2 -Gw -wd4244 -wd4267 -we4553 -Zi -O1 -Oi -Oy-    -Fdgenerated.pdb -FS  z:/build/build/src/third_party/prio/prio/encrypt.c
22:28:34     INFO -  encrypt.c
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2057: expected constant expression
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2466: cannot allocate an array of constant size 0
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(80): error C2133: 'key_bytes': unknown size
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(84): error C2057: expected constant expression
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(84): error C2466: cannot allocate an array of constant size 0
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(84): error C2133: 'spki_data': unknown size
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(281): error C2057: expected constant expression
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(281): error C2466: cannot allocate an array of constant size 0
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(281): error C2133: 'aadBuf': unknown size
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(319): error C2057: expected constant expression
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(319): error C2466: cannot allocate an array of constant size 0
22:28:34     INFO -  z:/build/build/src/third_party/prio/prio/encrypt.c(319): error C2133: 'aad_buf': unknown size
22:28:34     INFO -  z:/build/build/src/config/rules.mk:779: recipe for target 'encrypt.obj' failed
22:28:34     INFO -  mozmake.EXE[4]: *** [encrypt.obj] Error 2
22:28:34     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/build/src/obj-firefox/third_party/prio'
22:28:34     INFO -  mozmake.EXE[4]: *** Waiting for unfinished jobs....
22:28:34     INFO -  mozmake.EXE[4]: Entering directory 'z:/build/build/src/obj-firefox/config/external/icu/common'
22:28:34     INFO -  z:/build/build/src/obj-firefox/_virtualenvs/init/Scripts/python.exe -m mozbuild.action.cl  z:/build/build/src/vs2017_15.6.6/VC/bin/Hostx64/x86/cl.exe -Focwchar.obj -c  -DDEBUG=1 -DU_COMMON_IMPLEMENTATION -DUCONFIG_NO_TRANSLITERATION -DUCONFIG_NO_REGULAR_EXPRESSIONS -DUCONFIG_NO_LEGACY_CONVERSION -DU_USING_ICU_NAMESPACE=0 -DU_NO_DEFAULT_INCLUDE_UTF_HEADERS=1 -DU_CHARSET_IS_UTF8 -DU_HAVE_NL_LANGINFO_CODESET=0 -DU_DEBUG=1 -Iz:/build/build/src/config/external/icu/common -Iz:/build/build/src/obj-firefox/config/external/icu/common -Iz:/build/build/src/intl/icu/source/i18n -Iz:/build/build/src/obj-firefox/dist/include -Iz:/build/build/src/obj-firefox/dist/include/nspr -Iz:/build/build/src/obj-firefox/dist/include/nss -MD -FI z:/build/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -utf-8 -TP -nologo -w15038 -wd5026 -wd5027 -Zc:sizedDealloc- -wd4091 -wd4577 -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -arch:SSE2 -Gw -wd4251 -wd4244 -wd4267 -wd4800 -wd4595 -wd4065 -we4553 -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -GR- -Zi -O1 -Oi -Oy- -GR -wd4005 -wd4333 -wd4996    -Fdgenerated.pdb -FS  z:/build/build/src/intl/icu/source/common/cwchar.cpp
22:28:34     INFO -  cl : Command line warning D9025 : overriding '/GR-' with '/GR'

Comment 3

[Robert Helmer [:rhelmer]]

Attachment: bug 1485946 - skip building libprio on MSVC r?ted, r?dmajor

libprio does not currently build with MSVC (since it only supports
C90 as a compiler), this is being worked on upstream at https://github.com/mozilla/libprio/issues/17

As we are almost certainly not going to ship Firefox build with MSVC anymore,
let's disable this to get it working on this Tier-2 platform.

Comment 4

[Robert Helmer [:rhelmer]]

Comment on attachment 9004050 [details]
bug 1485946 - skip building libprio on MSVC r?ted, r?dmajor

So this mostly works, but it's breaking part of the DOM code on MSVC and I don't understand why:

z:\build\build\src\obj-firefox\dist\include\mozilla/dom/Promise-inl.h(142): error C2244: 'mozilla::dom::Promise::ThenWithCycleCollectedArgs': unable to match function definition to an existing declaration

So really I'd like feedback on this patch but I don't know if we support that with Phabricator :)

Comment 5

[Robert Helmer [:rhelmer]]

Here is the try run:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a00ec63484cd8d5b67a22cfe05fedb57438c8dc7&selectedJob=195925615

Comment 6

[Intermittent Failures Robot]

24 failures in 109 pushes (0.22 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-central: 24

Platform breakdown:
* windows2012-32: 12
* windows2012-64: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1485946&startday=2018-08-25&endday=2018-08-25&tree=all

Comment 7

[(Away)]

ThenWithCycleCollectedArgs is brand-new code from bug 1484373 and it's not impossible that it contains independent MSVC bustage, or, non-unified bustage that is triggered on your MSVC builds by changing the SOURCES in that configuration.

I would try a build with prio disabled for clang-cl as well, to see if changing the unification makes clang-cl hit the same error (and if it does, its error message will be much more verbose/useful to tracking this down.)

Comment 8

[(Away)]

Hmm, I think MSVC is trying to say that the header and implementation don't agree on this part:

> note: 'EnableIf<IsHandlerCallback<Callback,Args...>::value[snip]
> note: existing declarations
> note: 'EnableIf<mozilla::IsSame<already_AddRefed<mozilla::dom::Promise>,unknown-type>::value,[snip]

It seems like, in the .inl file, MSVC doesn't recognize IsHandlerCallback, maybe because it was defined in the class but referenced from outside the class?

Comment 9

[Robert Helmer [:rhelmer]]

(In reply to David Major [:dmajor] from comment #7)
> ThenWithCycleCollectedArgs is brand-new code from bug 1484373 and it's not
> impossible that it contains independent MSVC bustage, or, non-unified
> bustage that is triggered on your MSVC builds by changing the SOURCES in
> that configuration.
> 
> I would try a build with prio disabled for clang-cl as well, to see if
> changing the unification makes clang-cl hit the same error (and if it does,
> its error message will be much more verbose/useful to tracking this down.)

Thanks for looking into this, I kicked off a try run that disables Prio unconditionally (so SOURCES above should be changed in all configurations):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c86b043bd298491d2d7ba557edc863ee1d7a6f3&selectedJob=195995655

Yesterday I tried building with the above patch locally on macOS, but it worked OK there.

Comment 10

[Kris Maglione [:kmag]]

(In reply to David Major [:dmajor] from comment #8)
> Hmm, I think MSVC is trying to say that the header and implementation don't
> agree on this part:
> 
> > note: 'EnableIf<IsHandlerCallback<Callback,Args...>::value[snip]
> > note: existing declarations
> > note: 'EnableIf<mozilla::IsSame<already_AddRefed<mozilla::dom::Promise>,unknown-type>::value,[snip]
> 
> It seems like, in the .inl file, MSVC doesn't recognize IsHandlerCallback,
> maybe because it was defined in the class but referenced from outside the
> class?

That is... odd. It's interesting that it turns `EnableIf<Promise::IsHandlerCallback...>` into `EnableIf<IsHandlerCallback...>` but I don't know what to make of it.

I'll see if I can find some variant that it accepts.

Comment 11

[Intermittent Failures Robot]

64 failures in 2568 pushes (0.025 failures/push) were associated with this bug in the last 7 days.

This is the #28 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-central: 60
* try: 4

Platform breakdown:
* windows2012-64: 32
* windows2012-32: 32

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1485946&startday=2018-08-20&endday=2018-08-26&tree=all

Comment 12

[Nathan Froyd [:froydnj]]

Comment on attachment 9004050 [details]
bug 1485946 - skip building libprio on MSVC r?ted, r?dmajor

Nathan Froyd [:froydnj] has approved the revision.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df602a252b66
skip building libprio on MSVC r=froydnj

Comment 14

[Narcis Beleuzu [:NarcisB]]

Backed out for Linting failure on third_party/moz.build

Backout link: https://hg.mozilla.org/integration/autoland/rev/735ce953ca625bfa083658b463e47040481236c9
Log link: https://treeherder.mozilla.org/logviewer.html#?job_id=196147545&repo=autoland

Comment 15

[Robert Helmer [:rhelmer]]

(In reply to Narcis Beleuzu [:NarcisB] from comment #14)
> Backed out for Linting failure on third_party/moz.build
> 
> Backout link:
> https://hg.mozilla.org/integration/autoland/rev/
> 735ce953ca625bfa083658b463e47040481236c9
> Log link:
> https://treeherder.mozilla.org/logviewer.html#?job_id=196147545&repo=autoland

Thanks, updated the review request.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0400aff8c518
skip building libprio on MSVC r=froydnj

Comment 17

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/0400aff8c518

Comment 18

[Robert Helmer [:rhelmer]]

Hrm, it looks like we're not building Prio on any platform, now :/ (well I looked on macOS at least)

Any ideas? I am going to start looking, but could have sworn I tried locally..

Comment 19

[Kris Maglione [:kmag]]

(In reply to Robert Helmer [:rhelmer] from comment #18)
> Hrm, it looks like we're not building Prio on any platform, now :/ (well I
> looked on macOS at least)
> 
> Any ideas? I am going to start looking, but could have sworn I tried
> locally..

Presumably you need to return the `else:` here so you don't wind up returning `None` when you have `info` but the compiler isn't MSVC:

+# ==============================================================
+@depends(c_compiler)
+def libprio(info):
+    if info:
+      if info.type in ('msvc'):
+        return None
+    else:
+      return True

*However*. This is still super wrong, since ('msvc') is a string, not a tuple, so this will return true if `info.type` is 'm', 'ms', 'msv', 'msvc', 's,', 'sv', 'svc', 'v', 'vc', or 'c'.

Not that most of those are super likely to happen.

Comment 20

[Robert Helmer [:rhelmer]]

(In reply to Kris Maglione [:kmag] from comment #19)
> (In reply to Robert Helmer [:rhelmer] from comment #18)
> > Hrm, it looks like we're not building Prio on any platform, now :/ (well I
> > looked on macOS at least)
> > 
> > Any ideas? I am going to start looking, but could have sworn I tried
> > locally..
> 
> Presumably you need to return the `else:` here so you don't wind up
> returning `None` when you have `info` but the compiler isn't MSVC:
> 
> +# ==============================================================
> +@depends(c_compiler)
> +def libprio(info):
> +    if info:
> +      if info.type in ('msvc'):
> +        return None
> +    else:
> +      return True

Yeah, I just found that and re-opened the revision.

> 
> *However*. This is still super wrong, since ('msvc') is a string, not a
> tuple, so this will return true if `info.type` is 'm', 'ms', 'msv', 'msvc',
> 's,', 'sv', 'svc', 'v', 'vc', or 'c'.
> 
> Not that most of those are super likely to happen.

Argh python. Thanks.

Comment 21

[Robert Helmer [:rhelmer]]

Attachment: Bug 1485946 - fix MOZ_LIBPRIO build switch for non-MSVC platforms r?kmag

Made an errant else in the last one, and didn't do the tuple properly.

Comment 22

[Kris Maglione [:kmag]]

Comment on attachment 9005543 [details]
Bug 1485946 - fix MOZ_LIBPRIO build switch for non-MSVC platforms r?kmag

Kris Maglione [:kmag] has approved the revision.

Comment 23

[Robert Helmer [:rhelmer]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=51f6158770b7536e1ee1b318cb9bdf6a90bcd558

MSVC build is green, and I downloaded builds for macOS and Windows just to double-check locally.

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7a65516025af
fix MOZ_LIBPRIO build switch for non-MSVC platforms r=kmag

Comment 25

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/7a65516025af

Comment 26

[Intermittent Failures Robot]

8 failures in 2545 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 8

Platform breakdown:
* windows2012-32: 4
* windows2012-64: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1485946&startday=2018-08-27&endday=2018-09-02&tree=all