ENH: Create a place for "optimization" ufuncs

[seberg]

We have quite a few issues that request things like minmax, sincos or maybe gufuncs that "chain" very common operations, such as all_greater.

I do think there is merit in such functions but we are not likely to add them into the NumPy main namespace. (There was an old PR or branch by Julian I think where he added a bunch of similar ones such as multiply-4-numbers-and-add (which I guess is a SIMD intrinsic). (Seems I probably remembered this PR for linalg, where the initial version included a few fused ops)

I see two options for this:

1. If anyone wants to create these functions, I think we can definitely add them into their own repository/project in the NumPy organization.
2. It could make sense to have a np.lib.performance, at least for a subset of this type of functions.

There is little chance of adding a new functions such as this in the main namespace though, so this is to encourage anyone interesting in it to create a small additional package and then it will be easier to discuss about option 2. as well. That is my current angle on it.

[WarrenWeckesser]

Some related issues:

• first nonzero element (Trac #1673) #2269 first nonzero element
• gufunc to test for all elements equal along axis #8513 gufunc to test for all elements equal along axis (PR ENH: Adding logical gufuncs, ie all_equal #8528)
• ENH: minmax function #9836 ENH: minmax function
• Make np.argmax, np.argmin to return indexes and values at the same time #15623 Make np.argmax, np.argmin to return indexes and values at the same time
• ENH: Create the ability for fused operations (fused ufuncs or map_reduce) style #11622 Fast, memory efficient absolute value max

[seberg]

One thing about which to put into NumPy proper and which not: I think a good starting point would be to argue that anything could go into an np.lib.performance if it is also useful for NumPy itself. For examples some functions will be useful for np.linalg.norm.

[mattip]

I wonder if we could grow a "fused loop" ability, where you would say np.fuse(max, min)(args, ...) which would

• create an output array for each operation
• internally loop once over the values and do both operations on each value

This is complicated for SIMD loops.

[jakirkham]

Also am curious how other array libraries would extend or build on top of these. Though that is probably less important than the points already raised.

[seberg]

@mattip, I agree. I was using gh-11622 as an "issue" gather those, but maybe we should make a second one.

Fused ops, such as minmax make sense even without fused-loops. And some of them, might be OK to just have anyway as optimized-loops now, but replace later with a generic machinery. Things like all_greater are a bit tricky though, and I might prefer them outside of NumPy for now personally.

minmax might be a good "trial balloon", because we can use it internally in NumPy for histogram and ptp (and maybe more)