Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: parse xrefs differently with new xref format #90

Merged
merged 6 commits into from Jul 30, 2021

Conversation

dandhlee
Copy link
Collaborator

@dandhlee dandhlee commented Jul 29, 2021

Before you open a pull request, note that this repository is forked from here.
Unless the issue you're trying to solve is unique to this specific repository,
please file an issue and/or send changes upstream to the original as well.


Updating xref parser to accommodate the new and only xref format: <xref uid="uid">text</xref>.

We need to be careful on two parts:

    1. Carefully extracting only text portion to replace so GoogleDocstring doesn't complain on the leftover xref syntax
    1. Carefully parse through the docstring as we strip whitespace as well, handled by looking at one token ahead when necessary.

Fixes #89
Fixes googleapis/python-firestore#408

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to README are included in PR

@dandhlee dandhlee requested a review from tbpg July 29, 2021 16:49
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jul 29, 2021
@dandhlee
Copy link
Collaborator Author

Testing on Kokoro with TEST_PLUGIN=true option first

@dandhlee dandhlee marked this pull request as ready for review July 29, 2021 21:28
@dandhlee dandhlee requested a review from a team July 29, 2021 21:28
initial_index += summary_part.find("<xref")
original_type = parsed_text[initial_index:initial_index+(parsed_text[initial_index:].find('>'))+1]
original_type = parsed_text[initial_index:initial_index+(parsed_text[initial_index:].find('/xref>'))+6]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is 6? Can we make that a constant or something?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

initial_index += len(original_type)
original_type = " ".join(filter(None, re.split(r'\n| |\|\s|\t', original_type)))
safe_type = 'xref_' + original_type[6:-1]
# Extract text from "<xref uid="uid">text</xref>"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit late now, but we could also add this UID to the references section with however we want it to show up. Then, we wouldn't need to do any HTML parsing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll keep that in mind perhaps for non-Cloud or third party xrefs.

if arg_name not in summary_info[var_types[cur_type]] and ':raises' not in cur_type:
summary_info[var_types[cur_type]][arg_name] = {}
except KeyError:
raise KeyError(f"Encountered wrong formatting, please check docstring for {name}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this will work. What if there is a release with bad docs? We still need to generate docs for it. So, we should make our best guess and move on.

Copy link
Collaborator Author

@dandhlee dandhlee Jul 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repos should have a docs-presubmit check for PRs, which runs the docs job to see whether it successfully runs this plugin on the repo for sphinx-build or not. For example see googleapis/python-firestore#407

While it's usually not set to be a required check, this lets contributors know that docs job would fail or pass before triggering a release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Might need to update this depending on how back filling old libraries goes.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do have the Kokoro job set up that could try and run it for all versions, I'll try testing on that for older versions.

@dandhlee dandhlee requested a review from tbpg July 30, 2021 14:59
if arg_name not in summary_info[var_types[cur_type]] and ':raises' not in cur_type:
summary_info[var_types[cur_type]][arg_name] = {}
except KeyError:
raise KeyError(f"Encountered wrong formatting, please check docstring for {name}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Might need to update this depending on how back filling old libraries goes.

@dandhlee dandhlee merged commit 22485e8 into master Jul 30, 2021
@dandhlee dandhlee deleted the fix_docstring_parser branch July 30, 2021 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docs-presubmit failing on python-firestore GoogleDocstring does not handle new xref format well
2 participants