Is there a way to enable support for Python Cells in the tagbar? #759
Comments
|
Ad-hock ways of sectioning code that aren't part of the language itself won't be easy to handle. Not impossible but not easy. If there is a standardized way of doing this support could be added to Universal Ctags upstream and then we could easily add support for it. Keep in mind this plugin doesn't parse the actual content file at all, it just parses the output of a tag provider. If you provide tags, you can setup Tagbar to give you an interface to them. but are there any tag providers out there that produce anything for "cells"? |
|
Ok posted there - let's see what they say 😉 |
|
From the issue and pr it looks like this is a variant on the python language? Or is this the python language itself with the cell definition used in comments to logically break up the code? Standard PythonIf this is straight up python, we might have a problem. The diff --git a/autoload/tagbar/types/uctags.vim b/autoload/tagbar/types/uctags.vim
index c1715ba..25abf73 100644
--- a/autoload/tagbar/types/uctags.vim
+++ b/autoload/tagbar/types/uctags.vim
@@ -814,15 +814,18 @@ function! tagbar#types#uctags#init(supported_types) abort
\ {'short' : 'c', 'long' : 'classes', 'fold' : 0, 'stl' : 1},
\ {'short' : 'f', 'long' : 'functions', 'fold' : 0, 'stl' : 1},
\ {'short' : 'm', 'long' : 'members', 'fold' : 0, 'stl' : 1},
- \ {'short' : 'v', 'long' : 'variables', 'fold' : 0, 'stl' : 0}
+ \ {'short' : 'v', 'long' : 'variables', 'fold' : 0, 'stl' : 0},
+ \ {'short' : 'C', 'long' : 'cell', 'fold' : 0, 'stl' : 1},
\ ]
let type_python.sro = '.'
let type_python.kind2scope = {
+ \ 'C' : 'cell',
\ 'c' : 'class',
\ 'f' : 'function',
\ 'm' : 'function'
\ }
let type_python.scope2kind = {
+ \ 'cell' : 'C',
\ 'class' : 'c',
\ 'function' : 'f'
\ }New Language:If this is a new language which is based on python, then we would probably need to define a new language type and change the " IPythonCell {{{1
let type_ipythoncell = tagbar#prototypes#typeinfo#new()
let type_ipythoncell.ctagstype = 'IPythonCell'
let type_ipythoncell.kinds = [
\ {'short' : 'i', 'long' : 'modules', 'fold' : 1, 'stl' : 0},
\ {'short' : 'c', 'long' : 'cell', 'fold' : 0, 'stl' : 1},
\ {'short' : 'f', 'long' : 'functions', 'fold' : 0, 'stl' : 1},
\ {'short' : 'm', 'long' : 'members', 'fold' : 0, 'stl' : 1},
\ {'short' : 'v', 'long' : 'variables', 'fold' : 0, 'stl' : 0}
\ ]
let type_ipythoncell.sro = '.'
let type_ipythoncell.kind2scope = {
\ 'c' : 'cell',
\ 'f' : 'function',
\ 'm' : 'function'
\ }
let type_ipythoncell.scope2kind = {
\ 'cell' : 'c',
\ 'function' : 'f'
\ }
let type_ipythoncell.kind2scope.m = 'member'
let type_ipythoncell.scope2kind.member = 'm'
let types.ipythoncell = type_python
ScopeThe other thing to keep in mind for all this is scope... I'm not familiar with the ctags code, but the way tagbar uses ctags for the code hierarchy like this is mainly using the scope that ctags outputs marking when a tag scope ends. The tag scope begins on the line where the tag is found, and the tag scope ends based on the language parser in ctags. This is seen in the ctags output here for a #include <stdio.h>
typedef struct someStruct_s {
int s2_var1;
int s2_var2;
} someStruct_t;
int function1(int c) {
return (0);
}
int main(int arvc, char *argv[]) {
return (0);
}Without an |
|
@raven42 thanks for the detailed analysis 🤟 This is standard Python, i.e. it's not defined by the language itself, but it's a useful feature that a number of editors also support. Vim too through plugins such as vim-ipython-cell In that line, we didn't realize that using |
|
Ok, as @masatake pointed out it seems that it still can work with the $ cat /tmp/python-cell.ctags
--langdef=PythonCell{base=Python}
--kinddef-PythonCell=c,cell,cells
--regex-PythonCell=/^# %%[ \t]*(.*[^ \t])/\1/c/
# YOU CAN ADD MORE PATTERNS HERE.
$ cat /tmp/input.py
import numpy as np
from matplotlib import pyplot as plt
from scipy.io import wavfile
import os
# %% generate sound
f = 12000
fs = 44100
t = np.arange(0, 1, 1/fs)
sound = np.sin(2*np.pi*f * t)
# %% plot sound
plt.plot(t, sound)
# %% play sound
wavfile.write('sound.wav', fs, np.int16(sound * 2**15))
os.system('play sound.wav')
$ u-ctags --sort=no --fields=+'{language}' --extras=+'{subparser}' --options=/tmp/python-cell.ctags -o - /tmp/input.py
np /tmp/input.py /^import numpy as np$/;" I language:Python nameref:module:numpy
plt /tmp/input.py /^from matplotlib import pyplot as plt$/;" x language:Python nameref:unknown:pyplot
generate sound /tmp/input.py /^# %% generate sound$/;" c language:PythonCell
f /tmp/input.py /^f = 12000$/;" v language:Python
fs /tmp/input.py /^fs = 44100$/;" v language:Python
t /tmp/input.py /^t = np.arange(0, 1, 1\/fs)$/;" v language:Python
sound /tmp/input.py /^sound = np.sin(2*np.pi*f * t)$/;" v language:Python
plot sound /tmp/input.py /^# %% plot sound$/;" c language:PythonCell
play sound /tmp/input.py /^# %% play sound$/;" c language:PythonCellI hope this doesn't complicate things on your side too much 🤞 |
|
@gerazov based on that input, it looks like that is a new type of language though. It is based on python, but it looks like that definition is creating a new language definition called As a more general question, for a python script that has this cell architecture in the file, can it also have classes? Based on the previous comments, it sounds like this is just normal python, so classes would likely be allowed even with cell definitions in the file. Tagbar parsingFrom the tagbar perspective, it does not use the The ctags command as tagbar would execute it currently is defined like this: So in this instance it is forcing a language type of Cell Tree HierarchyI guess one of the main questions to ask is how do you envision the tagbar tree structure to look like? Are you meaning to have the variables / functions / classes and such all reside in a tree under the cell definition above it? Or do you just want a listing of all the cells? For the below discussions, lets look at the following example: import os
# %% Cell_1 // start of 'Cell 1' scope
def some_function(some_arg):
return
# %% Cell_2 // end of 'Cell 1' scope and start of 'Cell 2' scope
class some_class:
# %% Cell_2.1 // start of 'Cell 2.1' scope still inside 'Cell 2' scope
def some_class_function(another_arg):
return
# %% Cell_2.2 // 'end of 'Cell 2.1' scope, start of 'Cell 2.2' scope, but still inside 'Cell 2' scope
def another_class_function():
return
# %% Cell_3 // end of 'Cell 2.2' scope, end of 'Cell 2' scope, start of 'Cell 3' scope
some_value = "some string"Scoped Cell HierarchyThen there is the question of scope again for the cells. If we look at the example of a scope for a class, there can be multiple functions that are part of a class. But a function cannot be a member of another class. There could be the same function name in another class, but that is a unique tag. For cells though, can a cell span across multiple classes? Or can there be multiple cells within a class as seen in the example python code above? If I'm understanding the feature request correctly, would you be expecting a tagbar tree view like this? Would this be valid? and what would the scope of the different cells be? I could be way off here, but normal scope in python is defined by using indentation, so my assumption would be the scope would follow the same idea for cells as well as indicated in the above example with comments after the Maybe I am way off here (entirely possible)? From the little bit I've read about the cell structure in python, it is more used for how to execute a given block of code while running in an interactive mode. Would there ever be a need to have a cell inside a function? Most of the examples I've seen are all executing in global space rather than inside a function or class. So maybe this is a more specific use-case. Perhaps the scope is more defined in terms of a given cell starts at the comment containing the cell definition, and the scope goes to the next cell definition regardless of indentation. If this is the case, now this would post another problem with the tagbar hierarchy. If we consider the above example, how would we define the tree structure with the cells in there? Tagbar uses the scope information from the ctags output to determine the parent. If we look at the example above, the ctags output that tagbar parses would be as follows: So in this case, if we look at the tag for If tagbar doesn't have the parent identifier, it doesn't know how to build the hierarchy. So in the case of the cell definitions, tagbar would need that info with something like this (assuming a capital Note all the changes in the Unscoped Cell HierarchyWithout the tag scope output from ctags, tagbar cannot properly identify the tag hierarchy. So all existing tag definitions would need to have the proper tag scope setup so they point to the new cell tag. Without that tag scope, they would fall into a generic category tree like this: So in this case, the tree hierarchy would be a little different given the same code example above. All cells would be in their own root level node and you wouldn't be able to collapse the tree of everything inside that that cell, but only use it as a way to quickly see the defined cells and jump to those cell lines. Final thoughtsAnyway, sorry to make this such a long write up... I'm just trying to understand how best to try to make this work and some of the end cases like this. If there was a completely new language type, this might be easier... but I don't want to cause something to break for others. Maybe this could be simplified from the tagbar perspective if it was enabled via some configuration flag, so it would only parse the python cells if |
|
@raven42 that's an excellent write up highlighting the many problems and outlining possible solutions! 🤟 To iterate on a few things. Cells are mostly useful in scripts for executing/debugging parts of the code. You rarely use them inside function or class definitions, but it's not excluded as it could help debugging chunks of the code by allowing you to send them easily to the IPython console. They can either be mixed in between other tags based on location, or also in a tree of their own. I personally would prefer the former as I use Tagbar as sort of an outline of my file making it easy to navigate and to get a sense of the current cursor position. On the other hand, Tagbar's general approach appears to be the latter I guess. At the end both work, so which ever is more convininient to implement. Alternatively it could be specified via a flag, e.g. This is how Spyder handles your sample code (not that it should be the gold standard, but just as an example):
In Spyder cell hierarchy is handled via the number of Having cell tags controlled via a flag is a great idea 👍 I wouldn't impose them as the standard feature of the language. |
|
Thanks for the info @gerazov. Based on this, no matter what the ctags output would need to change to provide a different tag kind for the cell definitions so we aren't overloading the If that is done, then we should be able to implement the If you would like to see the So either way there is at least some change needed on the ctags side. At a minimum, the tag kind needs to change. Optionally scope can be provided. Please work with the ctags team to get this update, and once that is supported, I'd be happy to help update the tagbar code accordingly. Feel free to reference this discussion in the ctags issue/PR if needed. |
Cells are useful for structuring code and navigating through them with the tagbar would be very useful. They are defined in several ways, e.g.
# %%:Atm this code only shows variables in the tagbar.
There are some plugins that give cell support like
vim-ipython-supportthat could be useful?The text was updated successfully, but these errors were encountered: