Files.walkFileTree silently swallows NoSuchFileException

[lihaoyi]

package test.os

import java.nio.file.attribute.BasicFileAttributes
import java.nio.file.{Files, NoSuchFileException, Path, Paths, SimpleFileVisitor}

object OsTestMain {
  def main(args: Array[String]): Unit = {
    Files.walkFileTree(
      Paths.get("."),
      new SimpleFileVisitor[Path]() {
        override def visitFile(file: Path, attrs: BasicFileAttributes) = {
          throw new NoSuchFileException("Boom")
        }
      }
    )
  }
}

This fails loudly on Scala-JVM (stacktrace bellow), silently completes on Scala-Native without error. Normal Exceptions don't seem to get swallowed, so must be something specific to java.nio.file.NoSuchFileException

Exception in thread "main" java.nio.file.NoSuchFileException: Boom
	at test.os.OsTestMain$$anon$1.visitFile(OsTestMain.scala:12)
	at test.os.OsTestMain$$anon$1.visitFile(OsTestMain.scala:10)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2811)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2882)
	at test.os.OsTestMain$.main(OsTestMain.scala:10)
	at test.os.OsTestMain.main(OsTestMain.scala)

Found while trying to get com-lihaoyi/os-lib#257 working

[LeeTibbert]

lihaoyi

Thank you for reporting this and providing the reproducer. Sorry that this defect took your time.

I used the reproducer to evoke the presenting problem on my development system. A good first step.

Progress to date:

I think the "silently swallows NoSuchFileException" in the title is a reasonable starting hypothesis but
not what is happening here. I suspect the issue is that the expected Exception is not being generated in the first place.

JVM seems to "visit" the Path, as a file, before any "PreVisitDirectory". The recursive call in
the provided log file gives a hint:

at java.base/java.nio.file.Files.walkFileTree(Files.java:2811)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2882)

Scala Native processes the contents of a directory, doing the appropriate "preVisit" and "postVisit".

To me, the JVM "visit"-as-file-preVisit-as-directory" seems somewhere between astonishing and just
plain wrong, but then I coded neither the JVM nor the original SN code (I did do the stream conversion
following the prior implementation, perhaps I missed something.)

The JVM doc provides no "de jure" description of the to-me odd visitation order, so we are left with
JVM implementation as the "de facto" standard.

Next steps:

I am not sure of how much sand I have in my glass before I go offline again for a few weeks.

Is this blocking you or have you been able to bypass it, now that you have found it?

The next step is to figure out if the visit-as-file-then-pre-visit-as-directory (then visit again as content of
directory?) is a peculiarity of the initial recursive call or if it also happens for sub-directories (all directories).

This means writing and executing some study programs. Your reproducer has already sped up
that effort. This is one of those problems where it is easy to get a fix wrong and cause more problems
that the original.

[LeeTibbert]

1. The presenting reproducer in the base topic has the same behavior on SN and JVM
   when the code form PR fix: javalib SimpleFileVisitor#visitFileFailed now matches JVM #3889 is used. That PR has now passed CI, making it eligible
   for review & eventual merge.

2. A 7+ year old bug, good catch!

3. The recursion in the traceback in the base topic comes from the the 2 argument
   method of walkFileTree calling the 4 argument method. Obvious after the fact,
   but concerning when tracing.

4. lihaoyi, your hypothesis was correct: the Exception was being generated and was being
   swallowed, just not in an initially obvious place.

5. Although the code in the base topic will work after the PR is merged, I would like
   to keep this Issue open for a while. First I would like to convince myself that
   the logic in walkFileTree is reasonably correct, even if, in my opinion, contorted.

   I also want to convince myself that the correct things are being visited or not.
   Whilst debugging, I caught hints that the visitFileFailed was being called in SN
   where it was not being called on JVM.

6. As mentioned, I am going to be on hiatus for a number of weeks. Sorry for not
   being able to provide timely information and/or fixes during that period.

[LeeTibbert]

PR #3889 was merged awhile (weeks?) ago. I believe this PR can be closed.

lihaoyi
If there is still an issue, please let me know. Thank you.