os.path.realpath('notadir/', strict=True) doesn't raise

[barneygale]

Bug report

Bug description:

GNU coreutils realpath -e errors out when given a path to a file (not a directory) with a trailing slash:

$ realpath -e python/
realpath: python/: Not a directory

... but Python's os.path.realpath(..., strict=True) doesn't raise any error:

>>> os.path.isfile('python')
True
>>> os.path.isdir('python')
False
>>> os.path.realpath('python/', strict=True)  # should raise NotADirectoryError
'/home/barney/projects/cpython/python'
>>> os.path.realpath('python/.', strict=True)  # should raise NotADirectoryError
'/home/barney/projects/cpython/python'

CPython versions tested on:

3.12, 3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• GH-118289: Fix handling of non-directories in os.path.realpath() #118290

[nineteendo]

Here's a list of all invalid paths for posix.stat(). All of these should raise an error for posixpath.realpath() in strict mode:

• New file: newfile.txt -> FileNotFoundError (OK)
• Under a file: foo.txt/bar.txt -> NotADirectoryError (NOK)
• Too long file name: "a" * 256 -> OSError (OK)
• Too long path: "a/" * 512 -> OSError (OK)
• Too many levels of symlinks: "s/" * 33 -> OSError (NOK)

Should I make an issue for the symlinks?

[barneygale]

Should I make an issue for the symlinks?

It's deliberate behaviour - os.path.realpath() doesn't place a limit on the number of symlink traversals, unlike Linux or Mac.

Does it differ from realpath -e?

[nineteendo]

It's deliberate behaviour - os.path.realpath() doesn't place a limit on the number of symlink traversals, unlike Linux or Mac.

You mean the built-in function in macOS?

wannes@Stefans-iMac dirs % realpath s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s
realpath: s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s: Too many levels of symbolic links

Maybe a max_symlinks argument would make more sense? With None for the system default & -1 for no limit.

[novaTopFlex]

I have just tested the os.path.realpath() function on my different python3 versions and have gotten mostly the same issue.

If one is still on python3.9:
The strict argument is not an available argument in Python version 3.9.x.

If one is on python3.10, python3.11, python3.12, or python3.13:
The strict argument is recognized, but the code does not raise a NotADirectoryError.

[nineteendo]

ntpath.realpath() also raises an error, I'll try to check with coreutils as well.

>>> import os
>>> os.path.realpath('s/' * 64, strict=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen ntpath>", line 722, in realpath
OSError: [WinError 1921] The name of the file cannot be resolved by the system: 'C:\\Users\\wanne\\path-picker\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s\\s'

[nineteendo]

Hmm, only with '--logical':

wannes@Stefans-iMac dirs % grealpath -e s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s
/Users/wannes/path-picker/link-test/dirs
wannes@Stefans-iMac dirs % grealpath -L s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s
grealpath: s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s: Too many levels of symbolic links

[eryksun]

On POSIX systems, wouldn't it be reasonable for posixpath.realpath() in strict mode to try to match the behavior of POSIX realpath()? This fails with ELOOP if too many levels of symlinks are encountered. On Windows, WinAPI CreateFileW() fails with ERROR_CANT_RESOLVE_FILENAME (1921) in this case, and ntpath.realpath() in strict mode raises a corresponding OSError. That seems correct to me.

Linux:

import os, ctypes
pg = ctypes.CDLL(None, use_errno=True)
buf = (ctypes.c_char * 4096)()
pg.realpath.restype = ctypes.c_char_p
pg.realpath.argtypes = (ctypes.c_char_p, ctypes.c_char_p)
for i in range(41):
    os.symlink(f's{i+1}', f's{i}')
open('s41', 'w').close()

>>> os.path.realpath('s0', strict=True)
'/home/user/Temp/test/symlinks/s41'
>>> pg.realpath(b's0', buf)
>>> ctypes.get_errno() == errno.ELOOP
True
>>> os.strerror(errno.ELOOP)
'Too many levels of symbolic links'

[barneygale]

Would that be instead of, or in addition to, the existing symlink loop detection? The current code uses a seen dict that can grow to an arbitrary size. I'm guessing OSs use a counter because it's faster and uses less memory? (And presumably more resilient against DoS via long-but-not-looping chains of symlinks?)

[nineteendo]

Let's track this further in #118441.

[nineteendo]

How about secret symlinks? Should these be allowed in non-strict mode?

wannes@Stefans-iMac dirs % sudo ls -l secret-symlink
l---------  1 wannes  staff  44 Jun 30  2023 secret-symlink -> /Users/wannes/path-picker/link-test/dirs/dir
wannes@Stefans-iMac dirs % grealpath -m secret-symlink
/Users/wannes/path-picker/link-test/dirs/secret-symlink

>>> import posixpath
>>> posixpath.realpath("secret-symlink")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen posixpath>", line 435, in realpath
  File "<frozen posixpath>", line 495, in _joinrealpath
PermissionError: [Errno 13] Permission denied: 'secret-symlink'

[barneygale]

Looks like a bug to me!

[nineteendo]

OK, see #118447.