Why Symbol.to_python -> str ?

[mmatera]

This is something that still I do not understand, since v 1.0: except for (I guess) exactly 4 3 cases:

• Symbol`True -> True (bool)
• Symbol`False -> False (bool)
• Symbol`Null -> None
• Symbol`I -> 1 -> This depends on the number format (Machine complex vs Precision complex)

I do not see a reason to convert a Symbol object (which are Python objects) to another thing, like a str. Two Symbol s can be readily compared as a is b or a == b which are equivalent forms. So, for instance, why convert them to str and then compare string values? If we want the name of the symbol, we can just ask for it (a.get_name()).

Also, this method has a parameter (evaluation) that allows us to convert them into numbers if there is a numeric value associated. But again, I feel that this is misleading, and could be better achieved as

eval_N(a, evaluation).to_python()

[mmatera]

I asked this because this kind of conversion happens inside the GraphicsBox routines.

[rocky]

I don't understand the question, but I will make some general observations.

From the beginning, the difference between a Lisp-like interpreter string and a Lisp-like symbol has been muddled and muddied. It's one of those things like using "leaf" for "element" and "apply" for "eval" (and probably there are other things like that too).

It doesn't help that the term "Symbol" in WMA is overloaded. It can mean a character symbol, or a variable symbol (as in find "x" in the equation ...). Also the notion of a variable binding in the Mathics code is probably lacking or muddled, so the Mathics Symbol class is a bit odd.

We have been trying to unmuddy things. First we added Symbol definitions for a lot of the Built-in Symbols rather than write Symbol("xxx") everywhere. Next we disallowed the looseness of having a string in the Head name in (compound) Expression creation.

However there are way too many areas where strings are used where symbols would have been used traditionally in a Lisp-like interpreter.

I just chalk it up to inexperience and ignorance. But there is a lot of code like that, the WMA language has lots of details, if not also subtleties, and these subtleties have been coded in without describing what's going on. So you don't know if it was done that way because of ignorance or because there was a particular intent. And when it is of a particular intent, since you don't know what the intent is, it is hard to assess if there is a better way to do it.

So keeping this in mind, I have been trying to tread lightly. That said, I can say there have been places where I haven't fully appreciated what was going on and I have made mistakes here and there.

The trend though has been to use Lisp-like symbols (the Mathics Symbol class) when that is what is meant. That is, it has been more and more used when we need a fully-qualified name in the System namespace, and not the string of characters that are used to display that symbol.

It occurs to me if Lisp had called this kind of thing a name there might have been a lot less confusion.

[mmatera]

@rocky, thanks for your answer. I guess that this is also connected with the need of adding double quotes surrounding strings in String.to_python(): this was the way to distinguish a result coming from a Symbol from the one coming from a String. Then, we needed to add an option for avoiding the double quotes...

[mmatera]

Looking at the discussion in #551, I imagine a good reason to want that to_python just returns native builtin Python objects: for this kind of objects, it is faster to compute hash keys. So, if we have an expression like '{key1->val1, key2->val2,...}' (key1, key2, ... Symbols) , then this is mapped to
{"System`key1": val1, "System`key2": val2,...},
which has faster access than
{key1: val1, key2: val2,...}

isn't it?

[rocky]

isn't it?

For the most part, no. Or if that is they way you are thinking of this, then please stop.

Exactly what the intent of to_python() was initially is unclear and/or it has changed since the beginning because its intention was not understood by those who came after.

So let me declare my understanding and thoughts around intention here for to_python(). And unless there are objections, moving forward let's declare that....

The Mathics interpreter has to evaluate WMA expressions. It implements evaluation in Python; to_python() is there to get the expression close to something that is Python for it to work with without making lots of assumptions about the ever changing state of the Mathics-specific run-time environment.

Similar and related, Mathics uses SymPy to evaluate many expressions including symbolic expressions. For the most part, you can pass pure Python to SymPy and it understands that. Furthermore, SymPy does not understand Mathics-specific objects. SymPy also defines new objects of its own for Vectors, Matrices, Expressions, and Symbolic Variables. Of course, Mathics uses this and wants to use these. So there is a to_sympy() that assists this. (Right now it isn't that good at automatically detecting when to convert something to one of the SymPy-specific kinds of objects, but that is a different issue). So for the most part, SymPy could use to_python() provided it stays within standard and not Mathics-specific constructs.

We can also say something similar about mpmath and NumPy.

We also have PythonForm and SympyForm (which should be called SymPyForm) which are largely broken but purport to provide Python and SymPy equivalents for Mathics constructs when possible. These basically use to_python() and to_sympy()

Lastly, and most importantly lacking a clear reason why to_python() should stray from this logical, easy-to-describe and easy to think about behavior, we should keep it that way. This helps the overall design, comprehensibility, reasoning about the code, and simplifies maintenance because there is a simple and well-defined rule to follow. Just because in one place you need to do something, or one day you run into a problem that can be solved by changing the behavior slightly different should not be a reason to break an API.

Of course, these kinds of things do come up. And when it appears that there are a large number of places or there is some semantic reason for breaking the API then we should do so. One alternative when this kind of thing comes up is to use a different function. But I hesitate to even mention this because what happens then is there is a rush to add new largely nonsensical functions rather than looking deeper to see if there is some higher-level concept or framework to fit in these special cases.

Efficiency is related but it is not the primary concern here. The primary concern is clarity, simplicity and comprehensibility.
(Don't get me wrong, I am concerned about efficiency - but right now global order-of-magnitude efficiency not efficiency that complicates the code for very small benefit)

From what I have seen in the code, the lack of a clear understanding of interpreter concepts, a lack of a comprehensible API and class structure has been a major source of the slowness of the code. And oddly inefficiency is caused often as a second-order or indirect effect: we have all sorts of clown-car coding all over the place that we can't even begin to reason about the code or define interfaces and pieces for which we could start improving. It is like those movies where a clown is carrying a board on his shoulders turns his body and the board and slaps another clown doing something else and neither makes progress. So nothing gets done.

A big source of inefficiency is conversions all over the place, or we do a general Expression.evaluate() instead performing a simple Python method call because there haven't been API's that have been used consistently or, in many cases, defined.