Implement DataFrame.insert

[xinrong-meng]

ref #1929

Insert column into DataFrame at a specified location.

        >>> kdf = ks.DataFrame([1, 2, 3])
        >>> kdf.insert(0, 'x', 4)
        >>> kdf.sort_index()
           x  0
        0  4  1
        1  4  2
        2  4  3

        >>> from databricks.koalas.config import set_option, reset_option
        >>> set_option("compute.ops_on_diff_frames", True)

        >>> kdf.insert(1, 'y', [5, 6, 7])
        >>> kdf.sort_index()
           x  y  0
        0  4  5  1
        1  4  6  2
        2  4  7  3

        >>> kdf.insert(2, 'z', ks.Series([8, 9, 10]))
        >>> kdf.sort_index()
           x  y   z  0
        0  4  5   8  1
        1  4  6   9  2
        2  4  7  10  3

        >>> reset_option("compute.ops_on_diff_frames")

[itholic]

I'll take a more closer look after passing the tests :)

[codecov-io]

Codecov Report

Merging #1983 (a132eb1) into master (c38c96f) will decrease coverage by 0.00%.
The diff coverage is 94.11%.

@@            Coverage Diff             @@
##           master    #1983      +/-   ##
==========================================
- Coverage   94.59%   94.59%   -0.01%     
==========================================
  Files          50       50              
  Lines       11194    11210      +16     
==========================================
+ Hits        10589    10604      +15     
- Misses        605      606       +1     

Impacted Files	Coverage Δ
databricks/koalas/missing/frame.py	100.00% <ø> (ø)
databricks/koalas/frame.py	96.56% <94.11%> (-0.02%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c38c96f...a132eb1. Read the comment docs.

[ueshin]

I found some bugs:

>>> kdf = ks.DataFrame([1, 2, 3], index=[10,20,30])
>>> kdf.insert(1, 'z', ks.Series([8, 9, 10]))
>>> kdf
      0     z
0   NaN   8.0
1   NaN   9.0
10  1.0   NaN
2   NaN  10.0
30  3.0   NaN
20  2.0   NaN

>>> pdf = pd.DataFrame([1, 2, 3], index=[10,20,30])
>>> pdf.insert(1, 'z', pd.Series([8, 9, 10]))
>>> pdf
    0   z
10  1 NaN
20  2 NaN
30  3 NaN

or

>>> kdf = ks.DataFrame({('x', 'a'): [1, 2, 3]})
>>> kdf.insert(1, 'z', ks.Series([8, 9, 10]))
Traceback (most recent call last):
...
AssertionError: [('x', 'a'), ('z',)]

>>> pdf = pd.DataFrame({('x', 'a'): [1, 2, 3]})
>>> pdf.insert(1, 'z', pd.Series([8, 9, 10]))
>>> pdf
   x   z
   a
0  1   8
1  2   9
2  3  10

[ueshin]

Shall we revert this change?

[xinrong-meng]

May I ask why?

[xinrong-meng]

I was thinking kdf and self should share the same anchor.

[ueshin]

I mean this line # TODO: add frep and axis parqameter

[ueshin]

btw

I was thinking kdf and self should share the same anchor.

kdf could have a different anchor after assigning the value if the value is not the same anchor.

[xinrong-meng]

I see, may I get some help understand requires_same_anchor parameter of _update_internal_frame ?

[ueshin]

If it's False, the existing linked Series will see the updates even when the anchor changes; otherwise the link will be disconnected.
See #1592 for more detail.

[xinrong-meng]

Thanks for the reference!

No matter what requires_same_anchor is, the test case doesn't seem to pass https://github.com/databricks/koalas/pull/1983/files#diff-028bf26a42786beb47e6707fe34867dc720d3279ae4c942226abc3eb40f26eeaR104.
Would you give me a clue?

[ueshin]

Otherwise, LGTM.

[ueshin]

Thanks! merging.

[xinrong-meng]

@ueshin Thank you!