Tuple comprehension is not supported

[zhs628]

No description provided.

[tekknolagi]

For what it's worth, there is no such thing as a tuple comprehension. (x for x in y) is a generator expression.

[blueloveTH]

For what it's worth, there is no such thing as a tuple comprehension. (x for x in y) is a generator expression.

however, this is a bit hard to impl for some reason.

[tekknolagi]

Yes, generators are a big pain to implement. And they are slow if done the naive way :/

[qtnc]

Hello,

I have had an idea to implement generator expressions.
It doesn't look so hard, but maybe it's the inefficient "naive way" that you precisely wanted to avoid.

Let's take this python code as a start:

def f ():
  return all(x>=0 for x in range(1000000))

print(f())

Of course it will go through all numbers and finally print True.

The idea would be basically to internally transform this code into:

def f ():

  def generator ():
    for x in values:
      yield x
  
  return generator()

print(f())

Of course the function generator wouldn't be really accessible in the code, it's just there for illustration.

What do you think about it?
Is this the naive way?

In fact, the functions all and any, which are implemented in builtins of pocketpy, aren't very useful as such without generators.
Of course you can always write:

all([x>=0 for x in values])

But it is going to create a list with a million of items!

---

I have tried to implement my idea, and I don't think it's so naive.
If I disassemble f with the original CPython, I get this:

  4           0 LOAD_GLOBAL              0 (all)
              2 LOAD_CONST               1 (<code object <genexpr> at 0x00E22B78, file "test.py", line 4>)
              4 LOAD_CONST               2 ('f.<locals>.<genexpr>')
              6 MAKE_FUNCTION            0
              8 LOAD_GLOBAL              1 (range)
             10 LOAD_CONST               3 (1000000)
             12 CALL_FUNCTION            1
             14 GET_ITER
             16 CALL_FUNCTION            1
             18 CALL_FUNCTION            1
             20 RETURN_VALUE

If I disassemble the generator code alone in CPython (I need to do it separately), I get this:

1           0 LOAD_FAST                0 (.0)
>>    2 FOR_ITER                14 (to 18)
4 STORE_FAST               1 (x)
6 LOAD_FAST                1 (x)
8 LOAD_CONST               0 (0)
10 COMPARE_OP               5 (>=)
12 YIELD_VALUE
14 POP_TOP
16 JUMP_ABSOLUTE            2
>>   18 LOAD_CONST               1 (None)
20 RETURN_VALUE

If I disassemble f in pocketpy with my own implementation of the feature, I get this:

4          0   LOAD_GLOBAL               672 (all)
           1   LOAD_NULL                 0
           2   LOAD_FUNCTION             0 (<generator>)
           3   LOAD_NULL                 0
           4   CALL                      0
           5   CALL                      1
           6   RETURN_VALUE              0
           7   LOAD_NONE                 0
           8   RETURN_VALUE              0

Disassembly of <generator>:
4          0   LOAD_NONLOCAL             29659 (range)
           1   LOAD_NULL                 0
           2   LOAD_CONST                0 (1000000)
           3   CALL                      1
           4   GET_ITER                  0
           5   FOR_ITER                  12
           6   STORE_FAST                0 (x)
           7   LOAD_FAST                 0 (x)
           8   LOAD_INTEGER              0
           9   COMPARE_GE                0
           10  YIELD_VALUE               0
           11  LOOP_CONTINUE             5
        -> 12  NO_OP                     0
           13  LOAD_NONE                 0
           14  RETURN_VALUE              0

I don't feel it that bad. I have the impression to be reasonably close.
But maybe I missed something very important?

Interestingly, it looks like CPython pass range(1000000) to the generator function as a parameter, while in my case, I let the generator function call it.
I don't think it makes a big difference, though.

What are your thoughts?

Thank you for reading.

[blueloveTH]

In fact, if you are processing very large number of iterations, you can use yield or class with __iter__, not necessary to use generator expression.

[blueloveTH]

And in pkpy list comprehension is recommended though it creates a list. Because a function with yield, in most cases, is slower than listcomp or for loop.

[blueloveTH]

Finally, the idea is not about all/any. Our parser is not able to recognize generator expression for some history reasons.