1726433 - Port bug 1726117 - Upgrade docker images to Debian 11

Status: RESOLVED FIXED

(Thunderbird :: Upstream Synchronization, task)

Comment 1

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/09fbb39172f8
Port bug 1726117: Add Debian 11 docker images. rs=bustage-fix

Comment 2

[Rob Lemley [:rjl]]

Attachment: Bug 1726433 - Port bug 1726117: Add Debian 11 docker images. rs=bustage-fix

Port of D122830.

Comment 3

[Rob Lemley [:rjl]]

Attachment: Bug 1726433 - Port bug 1726117: Upgrade toolchain tasks to Debian 11 image. r=darktrojan

Port of D122833.

Comment 4

[Rob Lemley [:rjl]]

Attachment: Bug 1726433 - Port bug 1726117: Upgrade tasks using the debian10-amd64-build image to the debian11-amd64-build image. r=darktrojan

Port of D122835.

Depends on D123011

Comment 5

[Magnus Melin [:mkmelin]]

Hmm, maybe these are needed to unbust us? Will go ahead and land.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/3d6380f6f855
Port bug 1726117: Upgrade toolchain tasks to Debian 11 image. r=darktrojan
https://hg.mozilla.org/comm-central/rev/92d7d733cdee
Port bug 1726117: Upgrade tasks using the debian10-amd64-build image to the debian11-amd64-build image. r=darktrojan DONTBUILD

Comment 7

[Rob Lemley [:rjl]]

Attachment: WIP: Bug 1726433 - Stop using config variables set for Firefox to determine how to build librnp. rs=bustage-fix

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/eb1ca78b3646
Back out 3d6380f6f8552. rs-bustage-fix
https://hg.mozilla.org/comm-central/rev/368233737dd1
Stop using config variables set for Firefox to determine how to build librnp. rs=bustage-fix

Comment 9

[Geoff Lankow (:darktrojan)]

Attachment: Bug 1726433 - Revert to using a build image for comm_taskgraph test jobs. r=rjl

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/cc4fa7dc5e23
Backed out changeset 368233737dd1 to un-break the Windows build.
https://hg.mozilla.org/comm-central/rev/b66fe0bd4b3a
Revert to using a build image for comm_taskgraph test jobs. rs=bustage-fix

Comment 11

[Rob Lemley [:rjl]]

Attachment: Bug 1726433 - Fix Linux builds in CI. r=darktrojan

This is counter-intuitive and may be a problem later since librnp.so now links
to libc++ (statically) and to libstdc++ dynamically. In theory it's "ok"
since everything was compiled with libc++ headers via "-stdlib=libc++".
stdc++-compat.o gets linked in whether we want it or not when "MOZ_STDCXX_COMPAT"
is set, with no obvious way to override that behavior.

Comment 12

[Rob Lemley [:rjl]]

Attachment: Bug 1726433 - Fix Windows libotr builds. r=darktrojan

Adjust the path to libssp-0.dll so it's found.
A follow-up bug needs to fix the error handling in build-libotr.sh so that
when something like this happens, libotr jobs do not complete "successfully".

Depends on D123297

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/9f64006fd35d
Fix Linux builds in CI. r=darktrojan
https://hg.mozilla.org/comm-central/rev/0c112cd70641
Fix Windows libotr builds. r=darktrojan

Comment 16

[ISHIKAWA, Chiaki]

Thank you for making the try-comm-central build work again after so many days of mysterious errors.

Now I have a question regarding two things to which I want people to pay attention.

I found and realized the issue while trying to find the cause of compilation/link errors on the try-comm-central compilation farm.

1. Strange error reporting in the log file.

What I mean is this.
I tried to look for errors on the compilation farm.
I had to look for the errors in the try-comm-central compilation log since I did not see the errors locally at all.

There seems to have been errors/warnings produced by clang on the compilation farm.
But somehow the proper warnings were not recorded in the try-comm-central treeherder log.

Here is what I mean: The excerpt from
https://firefoxci.taskcluster-artifacts.net/Lr-DU4eZTyaER1cwFL_3oQ/0/public/logs/live_backing.log

rker/workspace/obj-build/security/rlbox'
[task 2021-08-21T07:34:00.307Z] 07:34:00     INFO -  security/rlbox/librlbox.so.wasm.o
[task 2021-08-21T07:34:00.309Z] 07:34:00     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang --sysroot /builds/worker/fetches/sysroot-x86_64-linux-gnu -std=gnu99 -o librlbox.so.wasm.o -c  -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fstack-clash-protection -ftrivial-auto-var-init=pattern -DDEBUG=1 -I/builds/worker/checkouts/gecko/security/rlbox -I/builds/worker/workspace/obj-build/security/rlbox -I/builds/worker/checkouts/gecko/third_party/wasm2c/wasm2c -I/builds/worker/checkouts/gecko/extensions/spellcheck/hunspell/glue -I/builds/worker/checkouts/gecko/extensions/spellcheck/hunspell/src -I/builds/worker/checkouts/gecko/gfx/graphite2/src -I/builds/worker/checkouts/gecko/media/libogg -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -include /builds/worker/workspace/obj-build/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -fcrash-diagnostics-dir=/builds/worker/artifacts -fno-strict-aliasing -fPIC -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -Os -fno-omit-frame-pointer -funwind-tables -Werror -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wformat-type-confusion -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wno-error=tautological-type-limit-compare -Wunreachable-code -Wunreachable-code-return -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wno-range-loop-analysis -Wenum-compare-conditional -Werror=non-literal-null-conversion -Wstring-conversion -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=free-nonheap-object -Wno-error=return-std-move -Wno-error=atomic-alignment -Wno-error=deprecated-copy -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Werror=implicit-function-declaration -Wno-psabi -Wno-error=unused-function -fexperimental-new-pass-manager  -MD -MP -MF .deps/librlbox.so.wasm.o.pp   librlbox.so.wasm.c
[task 2021-08-21T07:34:00.309Z] 07:34:00     INFO -  static void w2c___cxa_guard_abort(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.309Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.309Z] 07:34:00     INFO -  static void w2c___cxa_deleted_virtual(wasm2c_sandbox_t* const sbx) {
[task 2021-08-21T07:34:00.309Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.309Z] 07:34:00     INFO -  static u32 w2c___cxa_uncaught_exception(wasm2c_sandbox_t* const sbx) {
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -  static u64 w2c___cxxabiv1____getExceptionClass__Unwind_Exception_const__(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -  static void w2c___cxxabiv1____setExceptionClass__Unwind_Exception___unsigned_long_long_(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u64 w2c_p1) {
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -  static u32 w2c___cxxabiv1____isOurExceptionClass__Unwind_Exception_const__(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -  static u32 w2c___strerror_l(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u32 w2c_p1) {
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.310Z] 07:34:00     INFO -  static void w2c___cxa_finalize(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -  static void w2c___secs_to_zone(wasm2c_sandbox_t* const sbx, u64 w2c_p0, u32 w2c_p1, u32 w2c_p2, u32 w2c_p3, u32 w2c_p4, u32 w2c_p5) {
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -  static u32 w2c___wasilibc_register_preopened_fd(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u32 w2c_p1) {
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -  static u64 w2c___ftello_unlocked(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -  static void w2c___towrite_needs_stdio_exit(wasm2c_sandbox_t* const sbx) {
[task 2021-08-21T07:34:00.311Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static u32 w2c___fseeko_unlocked(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u64 w2c_p1, u32 w2c_p2) {
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static void w2c___toread_needs_stdio_exit(wasm2c_sandbox_t* const sbx) {
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -              ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static u32 w2c___nl_langinfo(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static u32 w2c___lctrans_cur(wasm2c_sandbox_t* const sbx, u32 w2c_p0) {
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static u32 w2c___isspace_l(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u32 w2c_p1) {
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static u32 w2c___isblank_l(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u32 w2c_p1) {
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.312Z] 07:34:00     INFO -  static u32 w2c___isalpha_l(wasm2c_sandbox_t* const sbx, u32 w2c_p0, u32 w2c_p1) {
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -  static u32 w2c___getf2(wasm2c_sandbox_t* const sbx, u64 w2c_p0, u64 w2c_p1, u64 w2c_p2, u64 w2c_p3) {
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -  static u32 w2c___unordtf2(wasm2c_sandbox_t* const sbx, u64 w2c_p0, u64 w2c_p1, u64 w2c_p2, u64 w2c_p3) {
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -             ^
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -  21 warnings generated.
[task 2021-08-21T07:34:00.313Z] 07:34:00     INFO -  gmake[4]: Leaving directory '/builds/worker/workspace/obj-build/security/rlbox'

Well, there were 21 warnings generated. All right.
However, the log doesn't say what the problems were: we don't know what the warnings were.
I wonder where I should report this.
(Or has it been addressed already?)

2. Rust warnings that hinders quick discovery of "real" errors.
   Actually this has been a pain in the neck for close to two years now on my local PC.
   Here we go.
   There are many warning messages produced while rust codes are compiled even locally. (I am using gcc with full -DDEBUG if it matters.)

The problem is that the compile-time warnings are related to some internal error/warning reporting mechanisms in the rust code and they have "error:" or "warning:" (ignoring case) as part of the message.
To wit, note "&error::Error" in the warning message below.
Excerpt from https://firefoxci.taskcluster-artifacts.net/OnVRhzabRF-xJwjDzTZfmg/0/public/logs/live_backing.log.

[task 2021-08-21T07:39:49.175Z] 07:39:49     INFO -  warning: trait objects without an explicit `dyn` are deprecated
[task 2021-08-21T07:39:49.175Z] 07:39:49     INFO -     --> /builds/worker/checkouts/gecko/third_party/rust/rust-ini/src/ini.rs:645:32
[task 2021-08-21T07:39:49.176Z] 07:39:49     INFO -      |
[task 2021-08-21T07:39:49.176Z] 07:39:49     INFO -  645 |     fn cause(&self) -> Option<&error::Error> {
[task 2021-08-21T07:39:49.176Z] 07:39:49     INFO -      |                                ^^^^^^^^^^^^ help: use `dyn`: `dyn error::Error`
[task 2021-08-21T07:39:49.176Z] 07:39:49     INFO -      |
[task 2021-08-21T07:39:49.176Z] 07:39:49     INFO -      = note: `#[warn(bare_trait_objects)]` on by default
[task 2021-08-21T07:39:49.177Z] 07:39:49     INFO -      = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in the 2021 edition!
[task 2021-08-21T07:39:49.177Z] 07:39:49     INFO -      = note: for more information, see issue #80165 <https://github.com/rust-lang/rust/issues/80165>
[task 2021-08-21T07:39:49.177Z] 07:39:49     INFO -  warning: 15 warnings emitted

or note the "std::error::Error::description" in the warning below from the same log file.

[task 2021-08-21T07:40:47.019Z] 07:40:47     INFO -  warning: trait objects without an explicit `dyn` are deprecated
[task 2021-08-21T07:40:47.019Z] 07:40:47     INFO -     --> /builds/worker/checkouts/gecko/third_party/rust/glob/src/lib.rs:294:32
[task 2021-08-21T07:40:47.019Z] 07:40:47     INFO -      |
[task 2021-08-21T07:40:47.020Z] 07:40:47     INFO -  294 |     fn cause(&self) -> Option<&Error> {
[task 2021-08-21T07:40:47.020Z] 07:40:47     INFO -      |                                ^^^^^ help: use `dyn`: `dyn Error`
[task 2021-08-21T07:40:47.020Z] 07:40:47     INFO -      |
[task 2021-08-21T07:40:47.021Z] 07:40:47     INFO -      = note: `#[warn(bare_trait_objects)]` on by default
[task 2021-08-21T07:40:47.021Z] 07:40:47     INFO -      = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in the 2021 edition!
[task 2021-08-21T07:40:47.022Z] 07:40:47     INFO -      = note: for more information, see issue #80165 <https://github.com/rust-lang/rust/issues/80165>
[task 2021-08-21T07:40:47.022Z] 07:40:47     INFO -  warning: use of deprecated associated function `std::error::Error::description`: use the Display impl or to_string()
[task 2021-08-21T07:40:47.023Z] 07:40:47     INFO -     --> /builds/worker/checkouts/gecko/third_party/rust/glob/src/lib.rs:291:20
[task 2021-08-21T07:40:47.023Z] 07:40:47     INFO -      |
[task 2021-08-21T07:40:47.024Z] 07:40:47     INFO -  291 |         self.error.description()
[task 2021-08-21T07:40:47.025Z] 07:40:47     INFO -      |                    ^^^^^^^^^^^
[task 2021-08-21T07:40:47.025Z] 07:40:47     INFO -      |
[task 2021-08-21T07:40:47.026Z] 07:40:47     INFO -      = note: `#[warn(deprecated)]` on by default
[task 2021-08-21T07:40:47.027Z] 07:40:47     INFO -  warning: 2 warnings emitted
[task 2021-08-21T07:40:47.029Z] 07:40:47     INFO -     Compiling serde_json v1.0.64

These are nuisances when I try to look for error lines in my local log using GNU Emacs editor.
It was MORE so when I tried to locate the real errors in the try-comm-central log file using web browser for a quick check.
Where should I report this?
Or is rust upstream doesn't care about this? [I think I have seen these warnings since the beginning of 2020, last year.]

---

Well, I am back to the runtime errors/warnings I see on the server farm. I could not get the compilation submission to work after repository corruption due to HG bug, and once this was fixed, this mysterious configuration/compilation/link errors. So I am behind close to two months.

Thank you again for making the compilation/linking back on track.

Comment 17

[Rob Lemley [:rjl]]

The remaining patch landed elsewhere: https://hg.mozilla.org/comm-central/rev/4a0ef7feba483f76491705ed29096be6e92e893a

Comment 18

[Rob Lemley [:rjl]]

Attachment: Bug 1726433 - Port bug 1726506: Cleanup unused debian10 images and packages. r=darktrojan

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/72166759ffaf
Port bug 1726506: Cleanup unused debian10 images and packages. r=darktrojan