Integral property in fixed-point arrays

[MarcT0K]

Hi,
I have several experiments using numpy-like arrays with secure integers and I am currently switching them to fixed-point numbers. However, this change turns out to be a bit challenging for one reason: the integral property of the arrays.

I've a few thousands lines of codes making various manipulations on numpy-like arrays including multiple np_transpose. This transpose checks that that the array integral property is not None. However, this assertion raises exceptions everywhere in my code. My experiments starts with arrays, whose integral property is either True or False depending on the context. However, there are several MPyC functions taking as input such arrays and outputting an array with integral == None. Hence, the code will fail on the next np_transpose. I can isolate some snippets if needed.

Here are examples of functions "forgetting" the integral property: np_sum, np_vstack, np_update

Then, I want to discuss to things:

1. Is the assert necessary in np_transpose? I am starting to understand the MPyC philosophy but I may miss some subtle details.
2. Is it normal that the integral property is "forgotten" by some MPyC functions? Intuitively, I would say that the integral can always be inferred from the integral property of the input. For example, in np_sum, the output should inherit the property of the input.

By the way, I can give a hand and prepare a dedicated PR if we agree on the (hypothetical) necessary changes.

[lschoe]

Hi Marc,

Yes, good point. The work for the integral attribute is ongoing anf not finished for all array operations. The assert in np_transpose and several other places are temporary, and should all be gone, once the integral attribute is supported throughout. These asserts are there to signal this omission. You can run things with asserts disabled (using -O as command line option for Python). Did you try that already? This may not work for you still because functions like np_sum do not propagate the integral attribute yet.

I can do a round of editing in the short term fixing a lot of these functions. But you can also do some and put it an a PR. I would suggest starting with just a few to see how things go.

[MarcT0K]

Hi,
I haven't used -0 because I also rely on some asserts in my experiments to do some sanity checks. Anyway, these problems were also a good occasion for me to refactor my code. I still struggle a little bit with the hstack, vstack, concatenate, column_stack, etc. especially when I juggle with 1D and 2D arrays at the same time.

you can also do some and put it an a PR. I would suggest starting with just a few to see how things go.

Ok, next week, I'll have some time to dedicate so I propose to do the following:

1. Unit tests to pin point the issue
2. Fix the issue for np_vstack, np_sum and np_update.

If we validate this first PR. I will prepare a second PR to extend it to all numpy-like functions.

[lschoe]

Yes, let's see. Actually, even do separate PRs for np_vstack, np_sum and np_update then we can discuss these one by one as well.

[lschoe]

Thx for the PR! I've merged it inwith the main branch, slightly rewriting the unit tests, have a look.

One thing to keep in mind for the PRs is to run pycodestyle mpyc and pycodestyle tests from the project's root directory. See the file setup.cfg for the settings used by pycodestyle. This passes normally without any errors or warnings.

[MarcT0K]

One thing to keep in mind for the PRs is to run pycodestyle mpyc and pycodestyle tests from the project's root directory.

Noted! Do you want me to submit a PR adding this linting check to the CI before the coverage?

[lschoe]

No, thx, 've left such checks out on purpose, don't want builds to fail for this kind of reason.