Closed Bug 1513429 Opened 10 months ago Closed 10 months ago

Upgrade image_builder image to something newer than Ubuntu 16.04

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(firefox-esr60 fixed, firefox66 fixed)

RESOLVED FIXED
Tracking Status
firefox-esr60 --- fixed
firefox66 --- fixed

People

(Reporter: gps, Assigned: gps)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

The image_builder Docker image is using Ubuntu 16.04.

Ubuntu 16.04 uses SQLite 3.11, which is ancient. It so happens that this SQLite is buggy w.r.t. multiple clients accessing repo data in parallel. I've managed to trigger db "corruption" with Mercurial working directory updates.

Let's upgrade the Docker image to something newer so we get a newer SQLite.
This appears to "just work."

While I would like to convert this image to Debian and make it
deterministic, that is more effect than I'm willing to invest at the
moment.

The impetus for this change is unblocking partial clones. Mercurial's
SQLite storage backend apparently hits a SQLite bug in version 3.11
of SQLite (what Ubuntu 16.04 runs) where SQLite complains about
database corruption when there are readers from multiple processes.
Ubuntu 18.04 is running SQLite 3.22 and doesn't exhibit the buggy
behavior.
I've gone ahead and scheduled this to autoland -- noteworthy is that dustin did comment in the bug that he feels python statically links sqlite so this upgrade may not fix the sqlite bug outright.
Pushed by jwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e52ecfe8a7b1
Upgrade image_builder to Ubuntu 18.04; r=dustin
Backed out changeset e52ecfe8a7b1 (Bug 1513429) for toolchain failure.

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=e52ecfe8a7b144b3079caa517f42550e17f571fa&selectedJob=217827046

Backout link: https://hg.mozilla.org/integration/autoland/rev/973d54f232af0074ebb4fd1a6eab4822d43e80c1

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=217827046&repo=autoland&lineNumber=1612

[task 2018-12-19T04:00:01.478Z] Install file: "Contrib/VPatch/oldfile.txt" as "/builds/worker/workspace/build/src/mingw32/share/doc/nsis/Examples/VPatch/oldfile.txt"
[task 2018-12-19T04:00:01.479Z] Install file: "Contrib/VPatch/patch.pat" as "/builds/worker/workspace/build/src/mingw32/share/doc/nsis/Examples/VPatch/patch.pat"
[task 2018-12-19T04:00:01.479Z] Install file: "Contrib/VPatch/Readme.html" as "/builds/worker/workspace/build/src/mingw32/share/doc/nsis/Docs/VPatch/Readme.html"
[task 2018-12-19T04:00:01.480Z] Install file: "Contrib/VPatch/VPatchLib.nsh" as "/builds/worker/workspace/build/src/mingw32/share/nsis/Include/VPatchLib.nsh"
[task 2018-12-19T04:00:01.485Z] scons: done building targets.
[task 2018-12-19T04:00:01.682Z] 
[task 2018-12-19T04:00:01.682Z] # --------------
[task 2018-12-19T04:00:01.682Z] 
[task 2018-12-19T04:00:01.682Z] cd $WORKSPACE/build/src
[task 2018-12-19T04:00:01.682Z] + cd /builds/worker/workspace/build/src
[task 2018-12-19T04:00:01.682Z] tar caf nsis.tar.xz mingw32
[task 2018-12-19T04:00:01.682Z] + tar caf nsis.tar.xz mingw32
[task 2018-12-19T04:00:03.485Z] 
[task 2018-12-19T04:00:03.485Z] mkdir -p $UPLOAD_DIR
[task 2018-12-19T04:00:03.485Z] + mkdir -p /builds/worker/artifacts
[task 2018-12-19T04:00:03.485Z] cp nsis.tar.* $UPLOAD_DIR
[task 2018-12-19T04:00:03.486Z] + cp nsis.tar.xz /builds/worker/artifacts
[fetches 2018-12-19T04:00:03.488Z] removing /builds/worker/workspace/build
Traceback (most recent call last):
  File "/builds/worker/bin/run-task", line 761, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/builds/worker/bin/run-task", line 756, in main
    shutil.rmtree(fetches_dir)
  File "/usr/lib/python3.5/shutil.py", line 480, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/lib/python3.5/shutil.py", line 418, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/usr/lib/python3.5/shutil.py", line 438, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.browser'
[taskcluster 2018-12-19 04:00:03.859Z] === Task Finished ===
[taskcluster 2018-12-19 04:00:05.984Z] Unsuccessful task run with exit code: 1 completed in 398.852 seconds
Flags: needinfo?(bugspam.Callek)
That's bug 1490573 and just requires a rerun or two.
Attachment #9030617 - Attachment description: Bug 1513429 - Upgrade image_builder to Ubuntu 18.04; r?dustin → Bug 1513429 - Upgrade image_builder to Ubuntu 18.04; r=dustin
Flags: needinfo?(bugspam.Callek)
Pushed by jwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2db40e6211b2
Upgrade image_builder to Ubuntu 18.04; r=dustin
https://hg.mozilla.org/mozilla-central/rev/2db40e6211b2
Status: ASSIGNED → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.