Minutes_2019_10_08

Attendees: Aaron, Ehsan, Pearu, Stuart, Todd, Stan

0. Feature Discussion

• Release status

  • backports ?
  • Siu to fix cupy array_interface compatibility
  • Will do final release as soon as this is fixed

• Improving test strategy

  • Public CIs are hitting time limits (60 min)
  • Turn around time on CI systems is hurting the ability to get feedback on PRs
  • Assumptions:
    • Public CI is essentially a smoke test
    • Public CI will never have the full spread of platform configurations
    • All we need it to do is give us an "is this reasonable?" feedback
  • Options:
    1. Create a spanning subset of tests: Has same coverage as full test suite, but may skip some type combinations or other unique scenarios.
       • Running test suite on a PR will always run modified test files + spanning subset
       • Advantages: all machines run the same tests
       • Disadvantages: have to keep recomputing the spanning subset
    2. Shard test suite across all platforms: Every test run on only one platform. Overcoverage should ensure some code paths tested on multiple platforms.
       • As above, running test suite on PR will always run modified test files everywhere + shard
  • Plans:
    • Switch from our custom unittest-derived test runner to using pytest
    • Write pytest plugin to detect which test files are changed relative to some branch (usually master)
    • Experiment with option #2 first. Maybe don't do #1?

• Copied from last week

  • Requests for 0.47 (last release for the year)
    • jitclass performance issues
    • llvm 9 trial
    • CTK libcudadevrt.a
    • CI needs to take 50% of current time
      • Val & Stu already looking at this
      • also checking Azure CI config to avoid wasting compute time
    • Caching:
      • transitive dependency
      • other issues: i.e. function as argument, with objmode
      • distributing cache
    • Immutable list and deprecating reflected list
    • Switch to pytest (see above)
    • Using Numba to generate LLVM/NVVM IR for different targets https://github.com/numba/numba/issues/4546
      • @overload for gpu

1. New issues

• #4666 - Interpreting Numba's PTX output
  • Siu is answering
• #4665 - Mod on uint64 near max values gives wrong result
  • PR #4667
• #4664 - numba problem
  • lack info
• #4663 - Accessing a numpy array defined in a module using the module name prefix fails in nopython mode
  • global lookup problem
• **** #4661 - Release candidate 0.46.0rc1 does not seem compatible with cupy
• #4659 - np.arange seems to be unsupported with exact dtype argument
  • keyword args issue with arange
• #4658 - A suggestion, an issue with int, and a strange issue with prange
  • PR opened for prange issue
• **** #4657 - Implement a pygments lexer for Numba IR
  • pretty-printer for more? ASM?
  • tries to co-op existing lexer
• #4654 - CUDA kernel function scalar parameter data type
• #4653 - induced >1d array iteration should be caught at type inference time
• **** #4647 - help diagnosing a GPU performance issue
  • need a range() rewrite for better perf

Already Closed

• #4662 - How can I get the numba0.46
• #4650 - This error is usually caused by passing an argument of a type that is unsupported by the named function
• #4649 - [Help] TypeError CUDA kernel or TypingError Numba type
• #4642 - cannot determine Numba type of <class 'function'>
• #4639 - Unsupported functionality was found in the code Numba was trying to compile.
• #4638 - ImportError: cannot import name 'jit' from 'numba'

2. New Open PRs

• #4667 - [WIP] Fix typo on urem implementation
• **** #4660 - Remove multiply defined variables from all blocks' equivalence sets.
• #4656 - fix sphinx build warning
• #4655 - fix sphinx build warning
• #4652 - Implement str.rfind()
• #4651 - Implement str.lower() and str.islower()
• #4648 - Handle 0 correctly as slice parameter.
• #4645 - Enable support for istitle() method for unicode string
• **** #4640 - test-suite instrumentation
  • Observations:
    • Function taking array-compatible arguments generate too many compiled versions
    • Top 5% of tests takes 30% of time

Already merged/closed

• #4643 - Bump to llvmlite 0.30

• #4641 - PR 4595 fixed against master

• #4646 - Change Log update for 0.46

• #4644 - Updates for llvmlite 0.30.0

4. Next Release: Version 0.46.0, RC=Sept 30

• Finish pending rewrite passes
• Document best practices for constructing new compiler pipelines
• Define autodiscovery system for Numba extensions (like numba_scipy or HPAT) that don't need direct user import
• Allow opt-in dispatching of functions by literal value
• Making caching aware of transitive dependencies
• Define declarative typing system for @overload (to be used in future releases)
• Numba Runtime C API for extensions to register reference-counted memory with the runtime.
• Start using new CI system in parallel with existing one
• Priority bug fixes:
  • Low performance of JIT method calls (requested by Pandas devs)
  • Others TBD