Simple file.py -> file.md use case

[kellyjonbrazil]

Hi there,

I'm still using pydoc-markdown version 2.1.0.post1 for my jc project because I just need a simple use-case of:

pydocmd simple jc+ > ../docs/readme.md

A while back it seemed that this use-case had broken with the newer versions and and it seems pydoc-markdown does more than I want it to. I just want a simple conversion for a file like the old behavior.

Is this possible with the newer versions? The reason I ask, is I started utilizing type-hinting and version 2.1.0.post1 doesn't seem to like type annotations:

$ pydocmd simple kbtest+
Building index...
Started generating documentation...
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/yapf/yapflib/pytree_utils.py", line 119, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib2to3/pgen2/driver.py", line 103, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib2to3/pgen2/parse.py", line 162, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=11, value=':', context=('', (1, 24))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/yapf/yapflib/pytree_utils.py", line 125, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib2to3/pgen2/driver.py", line 103, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib2to3/pgen2/parse.py", line 162, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=11, value=':', context=('', (1, 24))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/yapf/yapflib/yapf_api.py", line 183, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source)
  File "/usr/local/lib/python3.8/site-packages/yapf/yapflib/pytree_utils.py", line 131, in ParseCodeToTree
    raise e
  File "/usr/local/lib/python3.8/site-packages/yapf/yapflib/pytree_utils.py", line 129, in ParseCodeToTree
    ast.parse(code)
  File "/usr/local/Cellar/python@3.8/3.8.9/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 1
    get_help(parser_mod_name: str) -> None
                            ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pydocmd", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/pydocmd/__main__.py", line 252, in main
    loader.load_section(section)
  File "/usr/local/lib/python3.8/site-packages/pydocmd/loader.py", line 92, in load_section
    sig, _ = FormatCode(sig, style_config='pep8')
  File "/usr/local/lib/python3.8/site-packages/yapf/yapflib/yapf_api.py", line 186, in FormatCode
    raise errors.YapfError(errors.FormatErrorMsg(e))
yapf.yapflib.errors.YapfError: <unknown>:1:25: invalid syntax

I'm on python 3.8 and I updated my yapf library to the latest release. It seems like lib2to3 may be the culprit, but I'm not sure if the newer versions of pydoc-markdown call lib2to3 in a different way or if I can monkey patch the old version of pydoc-markdown to support type hints.

Any help would be appreciated, thanks!

[kellyjonbrazil]

I was able to get something similar by using this command line in the latest version (4.5.0)

pydoc-markdown -m jc > ../docs/readme.md

However the conversion is not the same as the older versions. For example, in the older version if I indented lines with four spaces in the docstrings I would consistently get raw monospaced code blocks. With the new version it doesn't reliably render these code blocks.

I didn't see in the documentation where I could configure the behavior of the markdown renderer. Could someone please point me in the right direction?

[kellyjonbrazil]

I'm getting closer. I was able to approximate the old behavior with the following command-line:

pydoc-markdown -m jc '{"processors": [{"type":"filter"},{"type":"pydocmd"}]}' > ../docs/readme.md

Basically it looks like I had to override the default configuration which includes the "smart" processor. In my case I want to hardcode the processor to pydocmd. I'll keep playing with it, but I think I'm getting the hang of it now.