Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parseToken array popping #13620

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

naomichi-y
Copy link
Contributor

@naomichi-y naomichi-y commented May 10, 2024

Description

Fixed an issue in the flat_object where array field names were incorrectly removed after processing the first object, leading to inaccurate key-paths in flattened JSON data.

Related Issues

#13351

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: naomichi-y <n.yamakita@gmail.com>
Signed-off-by: naomichi-y <n.yamakita@gmail.com>
Copy link
Contributor

❌ Gradle check result for 51d6d10: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 3862517: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Naomichi Yamakita <n.yamakita@gmail.com>
@naomichi-y
Copy link
Contributor Author

@msfroh
I tried to fix the issue where array fields disappear in flat_object Is this content correct?
#13259 (comment)

Copy link
Contributor

❌ Gradle check result for fe60f0d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Naomichi Yamakita <n.yamakita@gmail.com>
Signed-off-by: Naomichi Yamakita <n.yamakita@gmail.com>
Copy link
Contributor

❌ Gradle check result for c3adeb4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 7f7ee10: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@@ -74,8 +74,7 @@ public XContentParser parseObject() throws IOException {
return JsonXContent.jsonXContent.createParser(this.xContentRegistry, this.deprecationHandler, String.valueOf(jString));
}

private void parseToken(StringBuilder path, String currentFieldName) throws IOException {

private void parseToken(StringBuilder path, String currentFieldName, boolean popName) throws IOException {
Copy link
Collaborator

@reta reta May 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @naomichi-y , do you think we could refactor the method to use just Stack<String> to collect the current path instead of StringBuffer + boolean flag manipulation? (to be fair, it is pretty difficult to follow the current implementation and it is becoming even more complicated). Thank you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reta
I looked into the code a bit, and it seems that removing popName might be difficult.
The parseToken method parses the JSON recursively and removes trailing hierarchies from the field names depending on the type of the parent token.
However, I haven't been able to read the original code in its entirety, and I'm not very familiar with Java, so I might be misunderstanding something...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants