Document abstract methods

[shanemcq18]

Abstract methods – which have a signature but not a body – are currently not documented. It would be nice to be able to document them though, since they exist to define an interface for users to extend ("if you inherit from this class, you must write a function that...<precise explanation that should live in a docstring>").

The trouble is that these methods are special cases lexically because they aren't bookended by function and end. Here's an example.

classdef My_Class < handle
    % Docstring for the class.

    methods (Abstract)

        % How can we document this method?
        val = abstractmethod1(arg)
            % Perhaps a docstring here?

        % This is also not documented.
        [val1, val2] = abstractmethod2(arg1, arg2)

    end

    methods

        function [out] = publicmethod(arg)
            % This is documented.
            out = 2 * arg;
        end

    end
end

Now with this RST block somewhere,

.. autoclass:: My_Class
   :show-inheritance:
   :members:
   :undoc-members:

we get documentation for publicmethod() but not abstractmethod1() or abstractmethod2().

This was discussed a little way back in #28. Is this something that can be added without too much trouble? Adding function and end is not allowed for abstract methods, so it would have to be handled as a special case. It seems pretty natural to allow docstrings with this style:

[val1, val2] = abstractmethod2(arg1, arg2)
% This is the docstring.
%
% It may or may not continues for multiple lines.
% But watch out! It won't be followed by an ``end`` statement!

Thanks for writing this extension, it's a huge help for our project.

[joeced]

Sorry for the late reply and thanks for the report.

It's related to #44, which has been there for many years. It has not been easy to solve, but I'll try again next week.

[joeced]

This is just for reference. I tried the "textmate grammar" instead of pygments to converting some MATLAB code into tokens to compare the output.

Using your example class

Pygments output (removed consecutive Token.Text: ' ' for readability.

Token.Keyword: 'classdef'
Token.Text: ' '
Token.Name: 'My_Class'
Token.Text: ' '
Token.Operator: '<'
Token.Text: ' '
Token.Name: 'handle'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Comment: '% Docstring for the class.'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Keyword: 'methods'
Token.Text: ' '
Token.Punctuation: '('
Token.Name: 'Abstract'
Token.Punctuation: ')'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Comment: '% How can we document this method?'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Name: 'val'
Token.Text: ' '
Token.Punctuation: '='
Token.Text: ' '
Token.Name: 'abstractmethod1'
Token.Punctuation: '('
Token.Name: 'arg'
Token.Punctuation: ')'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Comment: '% Perhaps a docstring here?'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Comment: '% This is also not documented.'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Punctuation: '['
Token.Name: 'val1'
Token.Punctuation: ','
Token.Text: ' '
Token.Name: 'val2'
Token.Punctuation: ']'
Token.Text: ' '
Token.Punctuation: '='
Token.Text: ' '
Token.Name: 'abstractmethod2'
Token.Punctuation: '('
Token.Name: 'arg1'
Token.Punctuation: ','
Token.Text: ' '
Token.Name: 'arg2'
Token.Punctuation: ')'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Keyword: 'end'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Keyword: 'methods'
Token.Text.Whitespace: '\n'
Token.Keyword: '        \n        function'
Token.Text.Whitespace: ' '
Token.Text: '[out] '
Token.Punctuation: '='
Token.Text.Whitespace: ' '
Token.Name.Function: 'publicmethod'
Token.Punctuation: '('
Token.Text: 'arg'
Token.Punctuation: ')'
Token.Text.Whitespace: '\n            '
Token.Comment: '% This is documented.'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Name: 'out'
Token.Text: ' '
Token.Punctuation: '='
Token.Text: ' '
Token.Literal.Number.Integer: '2'
Token.Text: ' '
Token.Operator: '*'
Token.Text: ' '
Token.Name: 'arg'
Token.Punctuation: ';'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Keyword: 'end'
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Text.Whitespace: '\n'
Token.Text: ' '
...
Token.Text: ' '
Token.Keyword: 'end'
Token.Text.Whitespace: '\n'
Token.Keyword: 'end'
Token.Text.Whitespace: '\n'

Textmate Grammar Output with to_dict:

{
    "token": "source.matlab",
    "children": [
        {
            "token": "meta.class.matlab",
            "begin": [{"token": "storage.type.class.matlab", "content": "classdef"}],
            "end": [{"token": "storage.type.class.end.matlab", "content": "end"}],
            "children": [
                {
                    "token": "meta.class.declaration.matlab",
                    "children": [
                        {
                            "token": "entity.name.type.class.matlab",
                            "content": "My_Class",
                        },
                        {
                            "token": "punctuation.separator.lt.inheritance.matlab",
                            "content": "<",
                        },
                        {
                            "token": "meta.inherited-class.matlab",
                            "children": [
                                {
                                    "token": "entity.other.inherited-class.matlab",
                                    "content": "handle",
                                }
                            ],
                        },
                    ],
                },
                {
                    "token": "punctuation.whitespace.comment.leading.matlab",
                    "content": "    ",
                },
                {
                    "token": "comment.line.percentage.matlab",
                    "begin": [
                        {
                            "token": "punctuation.definition.comment.matlab",
                            "content": "%",
                        }
                    ],
                    "content": "% Docstring for the class.",
                },
                {
                    "token": "meta.methods.matlab",
                    "begin": [
                        {
                            "token": "keyword.control.methods.matlab",
                            "content": "methods",
                        },
                        {
                            "token": "storage.modifier.methods.matlab",
                            "content": "Abstract",
                        },
                    ],
                    "end": [
                        {
                            "token": "keyword.control.end.methods.matlab",
                            "content": "end",
                        }
                    ],
                    "children": [
                        {
                            "token": "punctuation.whitespace.comment.leading.matlab",
                            "content": "        ",
                        },
                        {
                            "token": "comment.line.percentage.matlab",
                            "begin": [
                                {
                                    "token": "punctuation.definition.comment.matlab",
                                    "content": "%",
                                }
                            ],
                            "content": "% How can we document this method?",
                        },
                        {
                            "token": "meta.assignment.variable.single.matlab",
                            "children": [
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "val",
                                }
                            ],
                        },
                        {"token": "keyword.operator.assignment.matlab", "content": "="},
                        {
                            "token": "meta.function-call.parens.matlab",
                            "begin": [
                                {
                                    "token": "entity.name.function.matlab",
                                    "content": "abstractmethod1",
                                },
                                {
                                    "token": "punctuation.section.parens.begin.matlab",
                                    "content": "(",
                                },
                            ],
                            "end": [
                                {
                                    "token": "punctuation.section.parens.end.matlab",
                                    "content": ")",
                                }
                            ],
                            "children": [
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "arg",
                                }
                            ],
                        },
                        {
                            "token": "punctuation.whitespace.comment.leading.matlab",
                            "content": "        ",
                        },
                        {
                            "token": "comment.line.percentage.matlab",
                            "begin": [
                                {
                                    "token": "punctuation.definition.comment.matlab",
                                    "content": "%",
                                }
                            ],
                            "content": "% Perhaps a docstring here?",
                        },
                        {
                            "token": "punctuation.whitespace.comment.leading.matlab",
                            "content": "        ",
                        },
                        {
                            "token": "comment.line.percentage.matlab",
                            "begin": [
                                {
                                    "token": "punctuation.definition.comment.matlab",
                                    "content": "%",
                                }
                            ],
                            "content": "% This is also not documented.",
                        },
                        {
                            "token": "meta.assignment.variable.group.matlab",
                            "begin": [
                                {
                                    "token": "punctuation.section.assignment.group.begin.matlab",
                                    "content": "[",
                                }
                            ],
                            "end": [
                                {
                                    "token": "punctuation.section.assignment.group.end.matlab",
                                    "content": "]",
                                }
                            ],
                            "children": [
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "val1",
                                },
                                {
                                    "token": "punctuation.separator.comma.matlab",
                                    "content": ",",
                                },
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "val2",
                                },
                            ],
                        },
                        {"token": "keyword.operator.assignment.matlab", "content": "="},
                        {
                            "token": "meta.function-call.parens.matlab",
                            "begin": [
                                {
                                    "token": "entity.name.function.matlab",
                                    "content": "abstractmethod2",
                                },
                                {
                                    "token": "punctuation.section.parens.begin.matlab",
                                    "content": "(",
                                },
                            ],
                            "end": [
                                {
                                    "token": "punctuation.section.parens.end.matlab",
                                    "content": ")",
                                }
                            ],
                            "children": [
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "arg1",
                                },
                                {
                                    "token": "punctuation.separator.comma.matlab",
                                    "content": ",",
                                },
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "arg2",
                                },
                            ],
                        },
                    ],
                },
                {
                    "token": "meta.methods.matlab",
                    "begin": [
                        {
                            "token": "keyword.control.methods.matlab",
                            "content": "methods",
                        }
                    ],
                    "end": [
                        {
                            "token": "keyword.control.end.methods.matlab",
                            "content": "end",
                        }
                    ],
                    "children": [
                        {
                            "token": "meta.function.matlab",
                            "begin": [
                                {
                                    "token": "storage.type.function.matlab",
                                    "content": "function",
                                }
                            ],
                            "end": [
                                {
                                    "token": "storage.type.function.end.matlab",
                                    "content": "end",
                                }
                            ],
                            "children": [
                                {
                                    "token": "meta.function.declaration.matlab",
                                    "children": [
                                        {
                                            "token": "meta.assignment.variable.output.matlab",
                                            "end": [
                                                {
                                                    "token": "keyword.operator.assignment.matlab",
                                                    "content": "=",
                                                }
                                            ],
                                            "children": [
                                                {
                                                    "token": "punctuation.section.assignment.group.begin.matlab",
                                                    "content": "[",
                                                },
                                                {
                                                    "token": "variable.parameter.output.matlab",
                                                    "content": "out",
                                                },
                                                {
                                                    "token": "punctuation.section.assignment.group.end.matlab",
                                                    "content": "]",
                                                },
                                            ],
                                        },
                                        {
                                            "token": "entity.name.function.matlab",
                                            "content": "publicmethod",
                                        },
                                        {
                                            "token": "meta.parameters.matlab",
                                            "begin": [
                                                {
                                                    "token": "punctuation.definition.parameters.begin.matlab",
                                                    "content": "(",
                                                }
                                            ],
                                            "end": [
                                                {
                                                    "token": "punctuation.definition.parameters.end.matlab",
                                                    "content": ")",
                                                }
                                            ],
                                            "children": [
                                                {
                                                    "token": "variable.parameter.input.matlab",
                                                    "content": "arg",
                                                }
                                            ],
                                        },
                                    ],
                                },
                                {
                                    "token": "punctuation.whitespace.comment.leading.matlab",
                                    "content": "            ",
                                },
                                {
                                    "token": "comment.line.percentage.matlab",
                                    "begin": [
                                        {
                                            "token": "punctuation.definition.comment.matlab",
                                            "content": "%",
                                        }
                                    ],
                                    "content": "% This is documented.",
                                },
                                {
                                    "token": "meta.assignment.variable.single.matlab",
                                    "children": [
                                        {
                                            "token": "variable.other.readwrite.matlab",
                                            "content": "out",
                                        }
                                    ],
                                },
                                {
                                    "token": "keyword.operator.assignment.matlab",
                                    "content": "=",
                                },
                                {
                                    "token": "constant.numeric.decimal.matlab",
                                    "content": "2",
                                },
                                {
                                    "token": "keyword.operator.arithmetic.matlab",
                                    "content": "*",
                                },
                                {
                                    "token": "variable.other.readwrite.matlab",
                                    "content": "arg",
                                },
                                {
                                    "token": "punctuation.terminator.semicolon.matlab",
                                    "content": ";",
                                },
                            ],
                        }
                    ],
                },
            ],
        }
    ],
}

The nice thing with "textmate-grammar" is that tokens are more rich. Still it will require quite a lot a work to convert to "textmate-grammar" parsing.