1523684 - L10n jobs are missing the setup stub - nightlies end up in exception

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Cosmin Sabou [:CosminS]]

Th push with exceptions: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=exception&revision=815f7149b30b27f79395d9a0155c1ebeb4fca8b5&selectedJob=224710490

Failure log: https://tools.taskcluster.net/groups/YHR6hk_aRtqq-gdYjA3wKQ/tasks/DpRFKOWnSIWqM2zVuYzYGw/runs/1/logs/public%2Flogs%2Fchain_of_trust.log#L130

https://taskcluster-artifacts.net/DpRFKOWnSIWqM2zVuYzYGw/1/public/logs/chain_of_trust.log

Callek said he will try to bisect this.

Pushlog from the latest green nightly to the exceptions: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d305772af471&tochange=815f7149b30b

Comment 1

[Justin Wood (:Callek)]

My current theory is Bug 1522354 - nalexander asked me to n-i him to get it in a queue -- I have a try push of a backout to test the theory: https://treeherder.mozilla.org/#/jobs?repo=try&revision=58aa8a88b7ab388bab1b1c59a0ca1c1e81cb3416

Comment 2

[Nick Alexander :nalexander [he/him]]

(In reply to Justin Wood (:Callek) from comment #1)

My current theory is Bug 1522354 - nalexander asked me to n-i him to get it in a queue -- I have a try push of a backout to test the theory: https://treeherder.mozilla.org/#/jobs?repo=try&revision=58aa8a88b7ab388bab1b1c59a0ca1c1e81cb3416

I don't understand why this line is present, but it smells bad:

if test "$OS_ARCH" = "WINNT"; then
  if ! test "$HAVE_64BIT_BUILD"; then
    if test "$MOZ_UPDATE_CHANNEL" = "nightly" -o \
            "$MOZ_UPDATE_CHANNEL" = "nightly-try" -o \
            "$MOZ_UPDATE_CHANNEL" = "aurora" -o \
            "$MOZ_UPDATE_CHANNEL" = "beta" -o \
            "$MOZ_UPDATE_CHANNEL" = "release"; then
      if ! test "$MOZ_DEBUG"; then
        if ! test "$USE_STUB_INSTALLER"; then
          # Expect USE_STUB_INSTALLER from taskcluster for downstream task consistency
          echo "ERROR: STUB installer expected to be enabled but"
          echo "ERROR: USE_STUB_INSTALLER is not specified in the environment"
          exit 1
        fi
        MOZ_STUB_INSTALLER=1
      fi
    fi
  fi
fi

It's ancient -- one wonders what's actually supposed to happen here now? Callek, can you try dropping it?

Comment 3

[Tom Prince [:tomprince]]

Attachment: Bug 1523684: Set `--target` to be a 32 bit target for L10n builds on win32; r?build

In Bug 1522354, the default for --host was changed to detect running on a 64bit host.
Since --target defaults to --host, this changed the default for --target
as well. This didn't affect builds, as they explictly set the --target in
build/win32/mozconfig.vs2017. Set it explicitly in L10n builds as well.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/58b77413fd91
Set `--target` to be a 32 bit target for L10n builds on win32; r=nalexander,Callek a=tomprince

Comment 5

[Tom Prince [:tomprince]]

https://hg.mozilla.org/mozilla-central/rev/58b77413fd91

Comment 6

[Justin Wood (:Callek)]

This was 1522354 for sure, my try push proved that -- Thanks Tom for writing the patch.

I'm leaving my n-i open so I can fix the devedition mozconfig tomorrow though (otherwise we'll hit this on next staging build/uplift)

Comment 7

[Intermittent Failures Robot]

52 failures in 478 pushes (0.109 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 52

Platform breakdown:

• windows2012-32: 52

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523684&startday=2019-01-29&endday=2019-01-29&tree=all

Comment 8

[Justin Wood (:Callek)]

Attachment: Bug 1523684: Set `--target` to be a 32 bit target for L10n builds on win32; r=nalexander

In Bug 1522354, the default for --host was changed to detect running on a 64bit host. Since --target
defaults to --host, this changed the default for --target as well. This didn't affect builds, as they
explictly set the --target in build/win32/mozconfig.vs2017. Set it explicitly in L10n builds as well.

This patch also fixes devedition

Comment 9

[Justin Wood (:Callek)]

Reopen for devedition

Comment 10

[Intermittent Failures Robot]

52 failures in 2628 pushes (0.02 failures/push) were associated with this bug in the last 7 days.

This is the #33 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 52

Platform breakdown:

• windows2012-32: 52

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523684&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d7772590ee4
Set `--target` to be a 32 bit target for L10n builds on win32; r=nalexander

Comment 12

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/0d7772590ee4