Different schema view from inner query and outer query

[michalwidera]

When simple query is issued like:
select one[1],one[2] stream one from a
But second query that is issued
select one[1] stream two from one+a
Will give inporper position in schema.
one[1] from stream one should be different from one[1] from stream two.
Now this leads to ambigous situtaion and errors in system reposnes.
More exaples needed.

Proposed solution - different, inner schema for inner queries and outer schema for outer queries.

[michalwidera]

refactor data storage

[michalwidera]

https://jamboard.google.com/d/1i9m0mHSzu57TnHi_FYUQJ27-HGMidEnDXb3iM0AunC0/viewer?f=0

[michalwidera]

Work on ANTLR4.

1. Work in stage area in Issue_17 branch
2. Sucessfull PR in Conan.io repository
3. Creating language from scratch ?

[michalwidera]

When simple query is issued like:
select one[1],one[2] stream one from a
But second query that is issued
select one[1] stream two from one+a

This shouldn't be allowed - this should appear as error.
Only:
select two[1] stream two from one+a
should be allowed

[michalwidera]

So far - issue_17 branch:

DECLARE a BYTE, b BYTE STREAM coreAB, 1 FILE '/dev/urandom'
DECLARE c BYTE, d BYTE STREAM coreCD, 0.5 FILE '/dev/urandom'

SELECT coreAB[0],coreAB[1],coreCD[0],coreCD[1] STREAM out1 FROM coreAB+coreCD
SELECT * STREAM out2 FROM coreAB+coreCD

Shows synchronized output

[michalwidera]

Issue_17 - dump

:STORAGE(0/1)   /home/michal/test/
out2(1/2)
        :- PUSH_STREAM(coreAB)
        :- PUSH_STREAM(coreCD)
        :- STREAM_ADD(0)
        Field_5:
                PUSH_ID(coreAB[0/1])
        Field_6:
                PUSH_ID(coreAB[1/1])
        Field_7:
                PUSH_ID(coreCD[0/1])
        Field_8:
                PUSH_ID(coreCD[1/1])
out1(1/2)
        :- PUSH_STREAM(coreAB)
        :- PUSH_STREAM(coreCD)
        :- STREAM_ADD(0)
        Field_0:
                PUSH_ID(coreAB[0/1])
        Field_1:
                PUSH_ID(coreAB[1/1])
        Field_2:
                PUSH_ID(coreCD[0/1])
        Field_3:
                PUSH_ID(coreCD[1/1])
coreCD(1/2)     /dev/urandom
        c:
        d:
coreAB(1/1)     /dev/urandom
        a:
        b:

[michalwidera]

Master dump:

coreCD(1/2)     '/dev/urandom'
        c:
        d:
out2(1/2)
        :- PUSH_STREAM(coreAB)
        :- PUSH_STREAM(coreCD)
        :- STREAM_ADD(0)
        Field_5:
                PUSH_ID(coreAB[0/1])
        Field_6:
                PUSH_ID(coreAB[1/1])
        Field_7:
                PUSH_ID(coreCD[0/1])
        Field_8:
                PUSH_ID(coreCD[1/1])
out1(1/2)
        :- PUSH_STREAM(coreAB)
        :- PUSH_STREAM(coreCD)
        :- STREAM_ADD(0)
        Field_0:
                PUSH_ID(coreAB[0/1])
        Field_1:
                PUSH_ID(coreAB[1/1])
        Field_2:
                PUSH_ID(coreCD[0/1])
        Field_3:
                PUSH_ID(coreCD[1/1])
coreAB(1/1)     '/dev/urandom'
        a:
        b:

This test case does not cover this issue properly.

[michalwidera]

Issue Explain Deatails
When we are testing folowing set of queries

DECLARE a BYTE, b BYTE STREAM core FROM 'data'
SELECT * STREAM p1 FROM core
SELECT a,b STREAM p2 FROM core
SELECT p3[0],p3[1] STREAM p3 FROM core

There is a problem with data picture seen from select or from outer source.
p2 is accessing directly from core and a,b elements are readed from core - but is should be readed from p2 context - materialized core from the given moment.
(here is this issue 17)
p3 currently does not compile - but it should.
We need full context from FROM clause with all the fields.
And query p2 and p3 sholuld read data from the context and not from core directly.
In a case of # opeartion problem is more important here.
Because there a,b could be ambigus and only context should remain accessible in SELECT clause algebraic operation list.

• External/Outer query - that creates outer context should see SELECT clause list effects.

• Internal computation from SELECT list should see full scan of FROM cluase context.

• Context should be hold in memory ?

• All result of SELECT clause opearions should remain on disk ?

[michalwidera]

update - "does not compile" - it compiles but executor fails here.

[michalwidera]

Done.