Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileMap with missing file(s) should optionally behave as if empty. #500

Open
blackwinter opened this issue Oct 2, 2023 · 1 comment
Open

Comments

@blackwinter
Copy link
Member

Whenever FileMap encounters a missing file (or the file cannot be read for other reasons) during access to the map (get() or keySet()), an exception is thrown and the isUninitialized flag remains unset (so it happens repeatedly on multiple accesses). For some use cases (e.g., lookup() in metafacture-fix) it might be preferable to ignore this error condition (maybe just log a warning) and continue as if the missing/unreadable file(s) were empty.


Caused by missing file map: lookup() calls map.get() which tries FileMap.init() leading to an exception every time. This then leaves the looked up value untouched even though delete: "true" was specified.

Originally posted by @blackwinter in hbz/lobid-resources#1906 (comment)

@blackwinter
Copy link
Member Author

Minimum viable solution (quietly ignoring errors):

diff --git metamorph/src/main/java/org/metafacture/metamorph/maps/FileMap.java metamorph/src/main/java/org/metafacture/metamorph/maps/FileMap.java
index 8462007e..621d0bbf 100644
--- metamorph/src/main/java/org/metafacture/metamorph/maps/FileMap.java
+++ metamorph/src/main/java/org/metafacture/metamorph/maps/FileMap.java
@@ -69,6 +69,7 @@ public final class FileMap extends AbstractReadOnlyMap<String, String> {
     private Matcher ignoreMatcher;
     private Pattern split = Pattern.compile("\t", Pattern.LITERAL);
     private boolean allowEmptyValues;
+    private boolean ignoreUnreadableFiles;
     private boolean isUninitialized = true;
     private int expectedColumns;
     private int keyColumn;
@@ -120,6 +121,18 @@ public void setIgnorePattern(final String ignorePattern) {
         this.ignoreMatcher = Pattern.compile(ignorePattern).matcher("");
     }
 
+    /**
+     * Sets whether to throw an exception upon encountering an unreadable file
+     * or ignore these files.
+     *
+     * <strong>Default value: false</strong>
+     *
+     * @param ignoreUnreadableFiles true if unreadable files should be ignored
+     */
+    public void setIgnoreUnreadableFiles(final boolean ignoreUnreadableFiles) {
+        this.ignoreUnreadableFiles = ignoreUnreadableFiles;
+    }
+
     /**
      * Sets a comma separated list of files which provides the {@link Map}.
      *
@@ -192,7 +205,9 @@ private void loadFile(final String file) {
             }
         }
         catch (final IOException | UncheckedIOException e) {
-            throw new MorphExecutionException("filemap: cannot read map file", e);
+            if (!ignoreUnreadableFiles) {
+                throw new MorphExecutionException("filemap: cannot read map file", e);
+            }
         }
     }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Backlog
Development

No branches or pull requests

1 participant