Reads from a file being written can succeed from page cache

[jamesbornholt]

Mountpoint for Amazon S3 version

mount-s3 1.4.1

AWS Region

No response

Describe the running environment

EC2 on AL2

Mountpoint options

mount-s3 bucket ~/mnt --allow-overwrite

What happened?

Running this program against an existing file:

use std::fs::OpenOptions;
use std::io::{Error, Write};
use std::os::unix::fs::FileExt;

fn main() -> Result<(), Error> {
    let filename = std::env::args()
        .nth(1)
        .expect("Please provide a filename as an argument");
    let mut file = OpenOptions::new()
        .read(true)
        .write(true)
        .truncate(true)
        .open(filename)?;

    file.write_all(&b"a".repeat(4097))?;

    let mut buffer = vec![0; 4097];
    let ret = file.read_at(&mut buffer, 0)?;

    println!("{}", ret);

    Ok(())
}

succeeds and prints 4096. It shouldn't succeed as our semantics says you can't read from a file that's open for writing, which this one is. This is a short read: we tried to read 4097 but only 4096 bytes succeeded, presumably because that first page is in the page cache.

The logs show that the OS tried to read starting at 4096, which must not have been cached.

I think maybe we need to go invalidate the page cache after a write or something. Or maybe just declare this is an OK behavior.

Relevant log output

2024-02-29T02:18:40.069748Z DEBUG fuser::request: FUSE(300) ino 0x0000000000000012 WRITE fh FileHandle(25), offset 0, size 4097, write flags 0x0
2024-02-29T02:18:40.084191Z DEBUG open{req=298 ino=18 pid=21150 name="README.md"}:put_object{id=91 bucket="bucket" key="README.md"}: mountpoint_s3_client::s3_crt_client: CRT request finished request_type=CreateMultipartUpload crt_error=None http_status=200 range=None duration=14.526954ms ttfb=Some(5.661692ms) request_id=010836fd9900018df2a81c4b050966e655bb080d
2024-02-29T02:18:40.084402Z DEBUG fuser::request: FUSE(302) ino 0x0000000000000012 READ fh FileHandle(25), offset 4096, size 4096
2024-02-29T02:18:40.084497Z  WARN read{req=302 ino=18 fh=25 offset=4096 size=4096 name="README.md"}: mountpoint_s3::fuse: read failed: file handle is not open for reads
2024-02-29T02:18:40.084563Z DEBUG fuser::request: FUSE(304) ino 0x0000000000000012 READ fh FileHandle(25), offset 4096, size 4096
2024-02-29T02:18:40.084655Z  WARN read{req=304 ino=18 fh=25 offset=4096 size=4096 name="README.md"}: mountpoint_s3::fuse: read failed: file handle is not open for reads