1659904 - UnixFile.open should be defined as variadic

Status: RESOLVED FIXED

(Toolkit Graveyard :: OS.File, defect)

Description

[Mike Hommey [:glandium]]

The definition of the function is variadic like the following on at least Linux and OSX:
int open(const char *, int, ...)

We're lucky that things are lined up in a way where it just works on all our current platforms, but it falls short on Apple Silicon macs because the ABI in that case is that the mode needs to end up on the stack which it doesn't if the function is not defined as variadic on the ctypes side.

Fortunately, variadic functions are supported by ctypes, although that's not documented. See https://bugzilla.mozilla.org/show_bug.cgi?id=554790#c22.

Additional complication, though: ctypes doesn't actually implement variadic functions correctly. Will file a seperate bug for the ctypes side of things.

Comment 1

[Mike Hommey [:glandium]]

Oh, I should mention, the result of this problem is that files opened with O_CREAT end up with garbage permissions, and we're lucky if they're readable at all after that.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1659904 - Define UnixFile.open as a variadic function.

That is how it is actually defined in fcntl.h, and thus how C code calls
it. It so happens that in many cases, it just works out fine because
variadic arguments end up at the same place as if they weren't variadic
in registers/stack, but that's not actually true on all platforms. Most
notably, that's not true on Apple Silicon.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2cb7fc096723
Define UnixFile.open as a variadic function. r=Yoric

Comment 4

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e8d3ae8df154
Backed out changeset 2cb7fc096723 for mochitest failures on test_osfile_back.xhtml. CLOSED TREE

Comment 5

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedTaskRun=fmqmnRzOTNCtJbFXD3smjw.0&resultStatus=testfailed%2Cbusted%2Cexception%2Crunning%2Cpending%2Crunnable&revision=2cb7fc096723dd93c5fd00574b4e53e68c6a8a28

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=313725726&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/e8d3ae8df154

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/891217d9ac80
Define UnixFile.open as a variadic function. r=Yoric

Comment 7

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/891217d9ac80