Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow usage of additional file extensions per parser. #3993

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

ammachado
Copy link
Contributor

What's changed?

Allow parsers to parse additional file extensions.

What's your motivation?

Parse Karaf's configuration files as properties.

Anything in particular you'd like reviewers to focus on?

No

Anyone you would like to review specifically?

No

Have you considered any alternatives or workarounds?

No

Any additional context

No

Checklist

  • I've added unit tests to cover both positive and negative cases
  • I've read and applied the recipe conventions and best practices
  • I've used the IntelliJ IDEA auto-formatter on affected files

@timtebeek timtebeek added the enhancement New feature or request label Feb 12, 2024
@knutwannheden
Copy link
Contributor

I like the idea! But I wonder if the execution context shouldn't instead allow registering alternative implementations for accept(), so that file extensions can also be removed (let's say someone doesn't want to parse Jenkinsfile files anymore). This would require some further refactoring, so that an alternative implementation can still optionally delegate to the default implementation. Thoughts on this?

@ammachado
Copy link
Contributor Author

Perhaps if we just add a file exclusion pattern, similar to the additionalExtensions implemented here, would work? ... accept(..) would check for the default extensions, then inclusions and exclusions.

rewrite-core/src/main/java/org/openrewrite/PathUtils.java Outdated Show resolved Hide resolved
rewrite-core/src/main/java/org/openrewrite/PathUtils.java Outdated Show resolved Hide resolved

public class ParsingExecutionContextView extends DelegatingExecutionContext {
private static final String PARSING_LISTENER = "org.openrewrite.core.parsingListener";
private static final String INCLUDE_PATTERNS = "%s.includePatterns";
private static final String EXCLUDE_PATTERNS = "%s.excludePatterns";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering a bit if we should adopt a gitignore similar mechanism where the patterns (include and exclude) are an ordered set and evaluated in order before deciding if a particular source is to be included or not. A strict separation of include and exclude patterns is less flexible. That being said, what files to parse and not parse almost certainly doesn't require the same level of flexibility as gitignore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I now understand that you did it this way so that the Parser#accept(Path) can remain as is and that the include and exclude patterns can be used to supply a "delta". Depending on how far we want to take this, that may indeed be a good compromise.

}

@NotNull
private Set<String> getPatternsFromSystemProperty(String key) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to not have the system property fallback. It seems like the client (e.g. the Maven build plugin) should be responsible for setting everything up.

@@ -96,16 +95,15 @@ public ParsingExecutionContextView setExcludePattern(Class<? extends Parser> par
}

public <T extends Parser> boolean accepts(T parser, Path path) {
for (String pattern : getExcludePattern(parser.getClass())) {
for (String pattern : getExcludePattern(parser.getParserClass())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new getParserClass() is an unfortunate implementation detail that surfaces in the API. I wonder if we can find a cleaner solution. The Parser.Builder#getDslName() looks like it would have been nicer, but unfortunately we don't have access to that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm providing a better implementation.


public class ParsingExecutionContextView extends DelegatingExecutionContext {
private static final String PARSING_LISTENER = "org.openrewrite.core.parsingListener";
private static final String INCLUDE_PATTERNS = "%s.includePatterns";
private static final String EXCLUDE_PATTERNS = "%s.excludePatterns";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I now understand that you did it this way so that the Parser#accept(Path) can remain as is and that the include and exclude patterns can be used to supply a "delta". Depending on how far we want to take this, that may indeed be a good compromise.

@timtebeek timtebeek linked an issue Mar 1, 2024 that may be closed by this pull request
@sambsnyd
Copy link
Member

sambsnyd commented Apr 2, 2024

It isn't clear to me that we need to make this extensible.
We're happy to add to the list of extensions for each parser. Do you have a file extension you'd like to accrue to one of the parsers by default? Do you have a scenario where hard-coding is insufficient or otherwise impossible?

@ammachado
Copy link
Contributor Author

This PR adds the option to include/exclude file patterns per parser. In my original use case, I have properties files that have .cfg extension instead of .properties, and I want to use the existing property recipes to refactor them.

@jimschubert
Copy link

I also described my use case in #4062. I have a yaml file without extension. To parse this as YAML required me to not only subclass YamlParser and a few other types, but I had to pass the file pattern as plain text matchers to the Maven plugin. That functionally has been deprecated, so it's not clear to me how I'd go about targeting my desired file mappings without this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Allow custom YAML filenames in YamlParser
5 participants