epilys

A minor gotcha in open(2)

A test I was writing in C was causing a panic in Rust code that was called via FFI. The panicking line of Rust code was something of the sort:

use nix::sys::stat::Mode;

fn foo(mode: libc::mode_t) {
  let mode: Mode = Mode::from_bits(mode).unwrap();
  ..
}

nix1 fails to handle a valid (I thought) mode value? There’s gotta be a simpler explanation: the mode value is invalid. Indeed looking at the C code:

char *file = "/tmp/file";
...
fd = open(file, O_WRONLY | O_CREAT | O_TRUNC);
...
example taken from POSIX open specification2

I had forgotten to specify the mode argument in open. However that’s legal behavior in POSIX, and it doesn’t define what happens when you call the variadic3 function open without the mode argument.

The glibc+Linux open(2) manual page with man 2 open explains what happens in their implementation:

The mode argument must be supplied if O_CREAT
or O_TMPFILE is specified in flags; if it is
not supplied, some arbitrary bytes from the
stack will be applied as the file mode [..]

Why am I not surprised! This should not be considered acceptable behavior for a standard library.


  1. Rust friendly bindings to *nix APIs https://crates.io/crates/nix↩︎

  2. https://pubs.opengroup.org/onlinepubs/007904875/functions/open.html↩︎

  3. variadic means it accepts a non fixed number of arguments, like printf. Its function signature is

    int open(const char *path, int oflag, ... );

    Without historical context, my guess this was to avoid having a different function when you want to specify the mode. Adding extra functions for extra arguments is an acceptable pattern in other libc functions, though.↩︎

return to index