draft: Use a SymPy Array not a Matrix for non-Expr

[cbm755]

It has mostly the same semantics as Matrix but can contain more general
things. There might be some issues about different shapes of empties
though... WIP on a fix for #1052 and #1055.

[cbm755]

@alexvong1995 here's a half-baked idea for Issue #1124: roughly when we have non-Expr to be put in a Matrix, we instead make an Array. This will be important for future SymPy as non-Expr in Matrix is deprecated.

[cbm755]

@alexvong1995 What do you think of this? Basically in a few places (probably not that many) we return Array instead of Matrix. Specifically we return Matrix greedily whenever everything is a Expr, else Array.

(It will need fixes upstream but I'm less worried about that.)

Maybe you can try doing this in the mat_rclist_ and other helper functions, just to get a feel for the idea.

[alexvong243f]

I think the the test failure is partly due to dbout being defined in inst/private/python_header.py (for non-Pythonic IPC) but not in inst/private/python_ipc_native.m (for Pythonic IPC).

I'm not sure what to do about it. It seems there's more code in inst/private/python_header.py than in inst/private/python_ipc_native.m.

[cbm755]

that could be true: I haven't looked at pythonic much recently. You can always comment out "dbout()" (it is just debugging). With Pythonic you can probably just print() instead.

Anyway, I think even concentrating on system-based, this idea is not complete.

[cbm755]

@alexvong1995 you can rebase and force push this on top of main (if you like).

[alexvong243f]

I think what we should do next is to make @sym/horzcat and @sym/vertcat Array-compatible. I haven't done any tests yet but I suspect all those [t t; f f] where t is sym(true) and f is sym(false) are causing many test failures.

[alexvong243f]

------- Original Message -------

On Monday, August 29th, 2022 at 12:04 AM, Colin B. Macdonald ***@***.***> wrote: @alexvong1995 you can rebase and force push this on top of main (if you like).

rebased onto main and force pushed as a7c44c0

…

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[cbm755]

while I think of it: I approve of this kind of "surface decrease" (having fewer functions that call pycall_sympy__) BUT it is important to be aware that there is a downside: namely more "roundtrips" between Python and Octave. This is most noticeable with ipc-system but even with popen2, there are additional copies being made as everything is serialized back-and-forth over the pipe.

Its analogous to HTTP where all objects keep getting converted back-and-forth to JSON.

No action required but its something to keep in mind: perfect software design here might blow-up the latency and IO costs.

[alexvong243f]

Interesting thoughts!

Normally when working with a single language, the classic solution is to use an (optimizing) compiler to optimize away all the redundant conversions. Unfortunately, this doesn't work for us because of language barrier: a compiler can't optimize across language boundary :(

---

Some of my thoughts:

First, pythonic may have already solved our problem! While I am not familiar with the internals of pythonic, perhaps using pythonic's py.... instead of pycall_sympy__ can avoid the needless conversions when going through a pipe. Of course, we should benchmark to see if it's indeed the case.

I believe this was the original future subproject but we changed it to the current (more urgent) one.

---

Another way is to create a octsmypy python library which contains one python function for each m-file function.

For example, say we want to implement m-file function @sym/transpose, @sym/vertcat and @sym/horzcat. Then our octsmypy python library should contain them as well. More precisely, we can do:

from octsympy.sym import transpose, vertcat, horzcat

To implement horzcat in terms of vertcat and transpose, we do it inside the octsmypy python library:

def horzcat(*XX):
    return transpose(vertcat(*[transpose(X) for X in XX]))

without the redundant conversions.

Finally, the m-file functions @sym/transpose, @sym/vertcat and @sym/horzcat are now thin wrappers over the corresponding octsympy python library functions, such as:

function h = vertcat (varargin)
    args = ...
    h = pycall_sympy__ ('from return octsympy.sym.vertcat(*_ins)', args{:});
end

and so on...

Perhaps this can be mentioned in the final report as possible future projects (?)

[cbm755]

I agree Pythonic offers possibilities. I think it currently still does conversion: a 'sym' is really just a struct of the 'srepr' (string representation) from SymPy, indep of IPC mechanism. But your 'xsym' idea would change that.

Agree about mentioing these things ad future projects.

Another thing to mention: CI pythonic runs are twice as slow as Popen2. I susect there migit be some 'low hanging fruit' in the Pythonic project.

[cbm755]

And just for the record: another alternative that requires very little effort is (a) being mindful of this issue and aiming for 1 (or a small number) of language crosses per function (b) balancing that against technical debt of code duplication etc. This is the approach we've been using so far.

[cbm755]

I think there was a way to turn an warning into an exception in Python. That might be useful when debugging here... like SymPyDeprecationWarning becomes an exception.