Please help me with combined Pydoc-Markdown and MKDocs #243

walter-weinmann · 2022-01-26T17:09:36Z

walter-weinmann
Jan 26, 2022

In my Python project, the documentation is created with MKDocs. I now want to create the API documentation with Pydoc-Markdown and add it to the MKDocs documentation. My directory layout for the documentation (docs) and the source files (src) can be seen here:

Now how do I adapt the MKDocs example in the Pydoc Markdown repository in order to

put the generated files with the API documentation inside the Docs directory ?
not to generate any other files ?

[[tool.pydoc-markdown.loaders]]
type = "python"
search_path = [ "../src" ]

[tool.pydoc-markdown.renderer]
type = "mkdocs"

[[tool.pydoc-markdown.renderer.pages]]
title = "API Documentation"
name = "index"
contents = [ "school.*" ]

In addition, what must the command line look like in my case?

pydoc-markdown ???

I would be very grateful if someone could help me with this.

NiklasRosenstein · 2022-02-07T21:56:28Z

NiklasRosenstein
Feb 7, 2022
Maintainer

Hi @walter-weinmann ,

Sorry for the late response. You can configure the directory in which the Mkdocs renderer generates the pages in using the output_directory and content_directory_name options. Pydoc-Markdown should not generate or overwrite an mkdocs.yml or any files other than the Markdown files if you don't specify the mkdocs_config option under the [tool.renderer] section.

[[tool.pydoc-markdown.loaders]]
type = "python"
search_path = [ "src" ]

[tool.pydoc-markdown.renderer]
type = "mkdocs"
output_directory = "."
content_directory_name = "docs"

[[tool.pydoc-markdown.renderer.pages]]
title = "API Documentation"
name = "index"
contents = [ "dcr.*" ]

To run Pydoc-Markdown, you don't have to supply any arguments. But you can pass -s,--server and -o,--open if you like, which will call mkdocs serve for you and open the browser.

$ pydoc-markdown -so

Can you give this a try and let me know how it works for you?

4 replies

walter-weinmann Feb 8, 2022
Author

Hi @NiklasRosenstein - thank you very much for your response.

Unfortunately, this configuration doesn't really work for me:

D:\SoftDevelopment\Projects\dcr>pydoc-markdown --dump

D:\SoftDevelopment\Projects\dcr>pydoc-markdown --dump -I src

D:\SoftDevelopment\Projects\dcr>pydoc-markdown --dump -I src\dcr
Traceback (most recent call last):
  File "C:\Software\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Software\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Software\Python\Python310\Scripts\pydoc-markdown.exe\__main__.py", line 7, in <module>
  File "C:\Software\Python\Python310\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Software\Python\Python310\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Software\Python\Python310\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Software\Python\Python310\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "C:\Software\Python\Python310\lib\site-packages\pydoc_markdown\main.py", line 339, in cli
    dump_module(module, sys.stdout)
  File "C:\Software\Python\Python310\lib\site-packages\docspec\__init__.py", line 531, in dump_module
    data = databind.json.dump(module, Module)
  File "C:\Software\Python\Python310\lib\site-packages\databind\json\__init__.py", line 117, in dump
    data = (mapper or new_mapper()).serialize(value, type_ or type(value), annotations=annotations, settings=options)
  File "C:\Software\Python\Python310\lib\site-packages\databind\core\mapper\objectmapper.py", line 106, in serialize
    return self.convert(Direction.serialize, value, type_hint, filename, pos, key, annotations, settings)
  File "C:\Software\Python\Python310\lib\site-packages\databind\core\mapper\objectmapper.py", line 84, in convert
    return ctx.convert()
  File "C:\Software\Python\Python310\lib\site-packages\databind\core\mapper\converter.py", line 120, in convert
    return converter.convert(self)
  File "C:\Software\Python\Python310\lib\site-packages\databind\json\modules\object.py", line 110, in convert
    return self._serialize(ctx, ctx.type)
  File "C:\Software\Python\Python310\lib\site-packages\databind\json\modules\object.py", line 37, in _serialize
    result[alias] = ctx.push(flat_field.field.type, value, name, flat_field.field).convert()
  File "C:\Software\Python\Python310\lib\site-packages\databind\core\mapper\converter.py", line 120, in convert
    return converter.convert(self)
  File "C:\Software\Python\Python310\lib\site-packages\databind\json\modules\optional.py", line 12, in convert
    return ctx.push(ctx.type.type, ctx.value, None, ctx.field).convert()
  File "C:\Software\Python\Python310\lib\site-packages\databind\core\mapper\converter.py", line 120, in convert
    return converter.convert(self)
  File "C:\Software\Python\Python310\lib\site-packages\databind\json\modules\object.py", line 110, in convert
    return self._serialize(ctx, ctx.type)
  File "C:\Software\Python\Python310\lib\site-packages\databind\json\modules\object.py", line 19, in _serialize
    raise ctx.type_error(expected=type_.schema.python_type)
databind.core.mapper.converter.ConversionError: expected `Docstring` to serialize `ObjectType(docspec.Docstring)`, got `str`
  Conversion trace:
    - $ (ObjectType(docspec.Module)) [None]
    - .docstring (OptionalType(ObjectType(docspec.Docstring))) [None]
    - $ (ObjectType(docspec.Docstring)) [None]

My configuration looks as follows:

D:\SoftDevelopment\Projects\dcr>pip freeze
atomicwrites==1.4.0
attrs==21.4.0
certifi==2021.10.8
charset-normalizer==2.0.11
click==8.0.3
colorama==0.4.4
databind.core==1.4.0
databind.json==1.4.0
dataclasses==0.6
Deprecated==1.2.13
distlib==0.3.4
docspec==1.2.0
docspec-python==1.2.0
docstring-parser==0.13
filelock==3.4.2
greenlet==1.1.2
idna==3.3
iniconfig==1.1.1
nr.fs==1.6.3
nr.optional==0.2.0
nr.parsing.date==1.0.3
nr.preconditions==0.0.4
nr.pylang.utils==0.1.3
nr.stream==0.2.3
nr.utils.re==0.3.1
packaging==21.3
pipenv==2022.1.8
platformdirs==2.4.1
pluggy==1.0.0
py==1.11.0
pydoc-markdown==4.5.0
pyparsing==3.0.7
pytest==7.0.0
PyYAML==5.4.1
requests==2.27.1
six==1.16.0
SQLAlchemy==1.4.31
toml==0.10.2
tomli==2.0.0
touch==2020.12.3
typing-extensions==3.10.0.2
urllib3==1.26.8
values==2020.12.3
virtualenv==20.13.0
virtualenv-clone==0.5.7
watchdog==2.1.6
wrapt==1.13.3

My pyproject.toml file:

[tool.bandit]

[tool.black]
line-length = 79

[tool.isort]
float_to_top = true
force_single_line = true
overwrite_in_place = true
profile = "black"

[tool.mypy]
check_untyped_defs = true
disallow_any_decorated = true
disallow_any_explicit = true
disallow_any_expr = false
disallow_any_generics = true
disallow_any_unimported = true
disallow_incomplete_defs = true
disallow_subclassing_any = true
disallow_untyped_calls = true
disallow_untyped_decorators = true
disallow_untyped_defs = true
follow_imports = "skip"
namespace_packages = false
no_implicit_optional = true
plugins = "sqlmypy"
pretty = true
show_column_numbers = true
show_error_codes = true
strict = true
strict_equality = true
warn_no_return = true
warn_redundant_casts = true
warn_return_any = true
warn_unreachable = true
warn_unused_configs = true
warn_unused_ignores = true

[[tool.pydoc-markdown.loaders]]
type = "python"
search_path = [ "src" ]

[tool.pydoc-markdown.renderer]
type = "mkdocs"
output_directory = "."
content_directory_name = "docs"

[[tool.pydoc-markdown.renderer.pages]]
title = "API Documentation"
name = "index"
contents = [ "dcr.*" ]

[tool.pydocstyle]

[tool.pylint]
# TBD
#min-similarity-lines = 12
#ignore-comments = true
#ignore-docstrings = true
#ignore-imports = true

[tool.pytest.ini_options]
console_output_style = "count"
markers = ["issue"]
pythonpath = "./src/dcr/"
testpaths = "./tests/"

NiklasRosenstein Feb 8, 2022
Maintainer

On mobile, but can you just try it without --dump? That is broken right now (there's an issue for it somewhere).

walter-weinmann Feb 9, 2022
Author

Running pydoc-markdown Version 4.5.0 gives:

D:\SoftDevelopment\Projects\dcr>pipenv run pydoc-markdown
[WARNING - pydoc_markdown.util.pages]: Page(title='API Documentation').contents has unmatched elements: dcr.*. Did you spell it correctly? Does a processor filter out this object?

New / modified files:

File `index.md` is now empty

File `mkdocs.yml` is now reduced to:

docs_dir: docs
nav:
- API Documentation: index.md
site_name: My Project

New file `generated.files.txt` contains:

md5 d41d8cd98f00b204e9800998ecf8427e docs\index.md

md5 ef04fc4b61aa7ab9e1d5baa0170dd2d4 mkdocs.yml

NiklasRosenstein Feb 28, 2022
Maintainer

@walter-weinmann I'm having some trouble to understand/reproduce your setup exactly. Could you link to your repository?

walter-weinmann · 2022-02-28T19:25:31Z

walter-weinmann
Feb 28, 2022
Author

https://github.com/KonnexionsGmbH/dcr

1 reply

NiklasRosenstein Feb 28, 2022
Maintainer

Thanks. You are pointing the Python loader to the src directory, which does contain dcr folder but that doesn't look like a package to it because there's no dcr/__init__.py. You have multiple options to fix this:

Add a dcr/__init__.py if dcr is intended to be a package (and dcr/dcr.py is supposed to be the dcr.dcr module)
If dcr/ is intended as a Python namespace package, you can explicitly tell the Python loader to look for a package named dcr (PEP 420 namespace packages are supported by docspec-python 2.0.0a1; as of right now you would need to use either the code in branch [WIP] Adopt new docspec and integrate with Novella #245 or wait for the Python 4.6.0 release which will be available soon)
i. This PR also fixes the --dump option that you haven't been able to use
If dcr/dcr.py is just the dcr module, then you can change the search path to src/dcr

Config for 2.

[[tool.pydoc-markdown.loaders]]
type = "python"
search_path = [ "src" ]
packages =  [ "dcr" ]

Config for 3.

[[tool.pydoc-markdown.loaders]]
type = "python"
search_path = [ "src/dcr" ]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Please help me with combined Pydoc-Markdown and MKDocs #243

{{title}}

Replies: 2 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Please help me with combined Pydoc-Markdown and MKDocs #243

walter-weinmann Jan 26, 2022

In my Python project, the documentation is created with MKDocs. I now want to create the API documentation with Pydoc-Markdown and add it to the MKDocs documentation. My directory layout for the documentation (docs) and the source files (src) can be seen here:

Now how do I adapt the MKDocs example in the Pydoc Markdown repository in order to

In addition, what must the command line look like in my case?

I would be very grateful if someone could help me with this.

Replies: 2 comments · 5 replies

NiklasRosenstein Feb 7, 2022 Maintainer

walter-weinmann Feb 8, 2022 Author

Hi @NiklasRosenstein - thank you very much for your response.

Unfortunately, this configuration doesn't really work for me:

My configuration looks as follows:

My pyproject.toml file:

NiklasRosenstein Feb 8, 2022 Maintainer

walter-weinmann Feb 9, 2022 Author

Running pydoc-markdown Version 4.5.0 gives:

New / modified files:

File index.md is now empty

File mkdocs.yml is now reduced to:

New file generated.files.txt contains:

NiklasRosenstein Feb 28, 2022 Maintainer

walter-weinmann Feb 28, 2022 Author

NiklasRosenstein Feb 28, 2022 Maintainer

Config for 2.

Config for 3.

walter-weinmann
Jan 26, 2022

Replies: 2 comments 5 replies

NiklasRosenstein
Feb 7, 2022
Maintainer

walter-weinmann Feb 8, 2022
Author

NiklasRosenstein Feb 8, 2022
Maintainer

walter-weinmann Feb 9, 2022
Author

File `index.md` is now empty

File `mkdocs.yml` is now reduced to:

New file `generated.files.txt` contains:

NiklasRosenstein Feb 28, 2022
Maintainer

walter-weinmann
Feb 28, 2022
Author

NiklasRosenstein Feb 28, 2022
Maintainer