Minutes_2019_08_20

Attendees: Aaron, Pearu, Stuart, Val, Stan, Siu, Ehsan, James

0. Feature Discussion

• Previous discussion that needs follow up
  • Adjust threadpool size https://github.com/numba/numba/issues/2713
    • does it satisfy the requirement if numba initialized max number of threads but can mask off threads? (James will gather more info)
      • 1st usecase: numba.set_num_threads(numthreads) to mask away threads. (Need to confirm if masking is acceptable)
      • 2nd usecase: forking parallel code is causing segfault. os.fork issue.
        • https://numba.pydata.org/numba-doc/dev/user/threading-layer.html#selecting-a-threading-layer-for-safe-parallel-execution
        • https://numba.pydata.org/numba-doc/dev/user/threading-layer.html#selecting-a-named-threading-layer
• PR 4425 is ready for review

1. New issues

• **** #4462 - parallel=True causes TypingError
  • need predicates, blocked on SSA?
• #4461 - Compilation is falling back to object mode WITH looplifting enabled
  • error is a bit of a tangential issue
  • general object mode issue
  • rest of code is not going to work in Numba
• #4460 - nb.deferred_type for jitclass does not work with any built-in method
  • Might be a use case we aren't going to support soon
  • May be some skew in support between jitclasses and regular functions
• #4458 - Assignment in numpy array using scalar array (either 0d, or 1d with length 1) fails
  • NumPy special cases
  • 1d len=1 might be the easier case to fix
  • 0d arrays have many potential issues in Numba
• #4457 - objmode block does not work inside a closure
  • maybe easy to fix? (good third issue)
• #4456 - objmode context manager has to be a global else it is not identified correctly
  • easy fix (good second issue)
• **** #4454 - Incorrect results on openblas for np.dot with out parameter
  • Need to detect input/output aliasing because MKL is currently covering for us. BLAS impls technically don't have to detect or handle aliasing.
  • lift maybe_alias from Todd's PR to solve the problem here.
• **** #4452 - Predictable RNG sequences under prange.
  • how to do RNG sensibly in threads?
  • Current state:
    • The impl: Duplicate numpy's RandomState in TLS. Each thread has unique RandomState.
  • What we may want:
    • let user define a list of RandomState.
    • let user know the "thread-id"?
  • Problems:
    • is that a valid usecase for prange?
    • exposing "thread-id" requires predicatable scheduling
  • Known approaches:
    • forward random-state
    • random number from hashes
  • Conlusion?
    • users will need a different RNG
    • Stan will respond
• #4451 - Add support for native python jitclass @staticmethod
  • Everyone loves namespaces
• #4450 - Rebuild Numba 0.45.1 packages with NumPy 1.17
  • pending AD release of NumPy 1.17
• **** #4449 - Incorrect output dtype for operations on arrays and scalars
  • Casting rules don't treat arrays and scalars differently
  • Suggestions:
    • make array dtype has a higher priority.
• #4448 - Document libraries that adopted __cuda_array_interface__
  • Good idea
• #4444 - can we use Numba on Phones?
  • Good idea? iOS is pointless, need to look at Android
• #4443 - numba.errors.InternalError
  • bumped into compiler limitation
  • need to get rid of StaticGetItem
• #4441 - Support more cuda intrinsic math functions
  • Good idea
• #4440 - Parfor analysis error on 0d array (arrayscalar)
  • More fun w/ 0d (see above)
  • asserts probably should not be in the compiler. Need to translate these into more precise error.
• #4439 - np.round optional arguments aren't optional
  • needs fixing, issue reporter looking at this

Already Closed

• #4459 - Failure when specify a list of jitclass instance

2. New Open PRs

• #4463 - [WIP] NamedTuple - Raises an error on non-iterable elements
• **** #4453 - An option to render the IR CFG into files during compilation.
• #4446 - Try this
  • Conclusion: Don't try this
• **** #4445 - [WIP] Make interpreter.py emit SSA form from bytecode
  • Putting on hold for now
  • Discovered several Parfors bugs
• #4442 - str.istitle() support for unicode string
  • Recreated from previously corrupted PR

Already merged

• #4455 - Clarify docs for typed dict with regard to arrays
• #4447 - Minor parallel performance tips edits

4. Next Release: Version 0.46.0, RC=Sept 30

• Finish pending rewrite passes
• Python 3.8 support
• Document best practices for constructing new compiler pipelines
• Define autodiscovery system for Numba extensions (like numba_scipy or HPAT) that don't need direct user import
• Allow opt-in dispatching of functions by literal value
• Making caching aware of transitive dependencies
• Define declarative typing system for @overload (to be used in future releases)
• Numba Runtime C API for extensions to register reference-counted memory with the runtime.
• Start using new CI system in parallel with existing one
• Priority bug fixes:
  • Low performance of JIT method calls (requested by Pandas devs)
  • Others TBD