Make Token inherit from Generic.

[Luis-omega]

Suggestion
The current Token class has signature of Optional[int] for column, line and positions. This means that you always need to

    a: int = token.line #type : ignore

Even when we already know that tokens must have non None attribute.

We can parameterize Token like:

_Token_info_type = TypeVar,("_Token_info_type", bound=Optional[int])

class Token(str,Generic[_Token_info_type]):

Describe alternatives you've considered
Until now this is forcing me to write

line = token.line #type : ignore 
column = token.column #type : ignore 
.
.
.
return SomeClass(...,line,column,...)

Additional context
I would like to write the above code as

return SomeClass(...,token.line,token.column,...)

Using #type : ignore works but using a formatter like black sometimes put in the same line two assignations while creating a object, including one using a toke.line or similar and the type ignore would hide bugs in the other assignation (happened 3 times to me until now).
Particularly this can happen inside a transformer method.

I don't think this would affect the rest of the code and would like to submit a pr for this.

[MegaIng]

The reason the signature is the way it is is because the line attributes might be None. If you want to write typesafe code, you will need to check for this. If you are checking for this and mypy can't figure out that you are doing the check, that would be a problem on mypy's side, not something we can fix.

[Luis-omega]

I thought something like that, this particularly can happen if you inject tokens after the lexer stage. But if you are not injecting tokens, then you end with every function on a transformer following this pattern

def some_rule(...,token:Token,...)->SomeType:
line = token.line #type:ignore

Are there other cases apart from token injection that can make the line a None while column or position isn't?

Otherwise the transformers code can be wrote like:

def some_rule(...,token:Token[int],...)->SomeType:
line = token.line

When you know you are not injecting tokens, and if you are, you can explicitly state that you are generating tokens without information by using Token[None]