Closed Bug 1612035 Opened 2 years ago Closed 2 months ago

lucet does not produce reproducible .so files

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox92 fixed)

RESOLVED FIXED
92 Branch
Tracking Status
firefox92 --- fixed

People

(Reporter: tjr, Assigned: glandium)

References

(Blocks 2 open bugs)

Details

(Whiteboard: [tor 33488])

Attachments

(2 files)

Attached file diffoscope.html.gz

As shown in the attached diffoscope output, it looks like at the very minimum the symbol ordering in the resulting library is not consistent.

(The diffoscope is 15.5MB, with an attachment limit of 10MB, so I gzipped it.)

Capturing some of my offline conversations with Tom Ritter for @Georg

My suspicion is that either

  1. lucet does not actually guarantee order of output symbols. If so, we would need to fix lucet which may be non trivial
  2. lucet does not guarantee order of output of private wasm symbols as part of generation of .so (we add a linker flag to explicitly expose all symbols). This one is more unlikely, but the fix would need me to see what functions are being marked public when clang generates the wasm module from graphite source

Lucet/cranelift uses some rust packages faerie, object and goblin to interact with the binary files. Searching for these strings in the lucet compiler repo is probably a good starting point. Last thing worth noting is that code may have changed in the upstream version as compared to the fork. I think it may be better to implement the change against upstream and I can backport this

Upstream: https://github.com/bytecodealliance/lucet
Fork: https://github.com/PLSysSec/lucet_sandbox_compiler

Tom Ritter: My current plan is to wait till we get back, ask glandium for his objdir patches, run it with them and see if that grants any insights

Before playing with the object files, it's also worth checking if the .wasm file produced by clang (which is then compiled with lucet) is identical on both builds

(In reply to Shravan Narayan from comment #1)

[snip]

Tom Ritter: My current plan is to wait till we get back, ask glandium for his objdir patches, run it with them and see if that grants any insights

Before playing with the object files, it's also worth checking if the .wasm file produced by clang (which is then compiled with lucet) is identical on both builds

Just to reply to that part: all the *.wasm files (including libgraphitewasm.so.wasm) match for me. Only the .so is different.

Whiteboard: [tor 33488]
Assignee: nobody → mh+mozilla
Depends on: 1720828

Now that the rlbox library is built with wasm2c instead of lucetc, it is
reproducible.

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/157802317459
Remove reproducibility exception for rlbox library. r=firefox-build-system-reviewers,andi DONTBUILD
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 92 Branch
You need to log in before you can comment on or make changes to this bug.