Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding the list of APIs we won't support in Koalas to docs. #1270

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Adding the list of APIs we won't support in Koalas to docs. #1270

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0458fc8
Add missing APIs to docs
itholic Feb 10, 2020
a7cb7dc
Add missing APIs to docs
itholic Feb 10, 2020
ae03177
Restore unnecessary change
itholic Feb 10, 2020
e371bb3
Restore unnecessary change
itholic Feb 10, 2020
60aa5b7
plan -> planned
itholic Feb 10, 2020
4c4ca9a
Add Example
itholic Feb 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
49 changes: 49 additions & 0 deletions docs/source/user_guide/faq.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,52 @@ lot longer (in the order of days)
2. Koalas takes a different approach that might contradict Spark's API design principles, and those
principles cannot be changed lightly given the large user base of Spark. A new, separate project
provides an opportunity for us to experiment with new design principles.

What is the list of APIs that are not planned to support in Koalas?
-------------------------------------------------------------------

Koalas doesn't support several APIs that may cause memory issues mostly due to the size of the data.

For example, implementing and using `DataFrame.values` in Koalas can cause all data belonging to the

DataFrame to be loaded into the driver's memory, causing memory errors like OOM.

The following is a list of APIs that Koalas doesn't plan to support.

- DataFrame
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we automatically generate this list and the APIs we didn't implement yet?

Copy link
Contributor Author

@itholic itholic Feb 11, 2020 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay let me try,

and IMO, i think we'd better separate the list into Not yet implemented and Not will be implemented if we gonna manage the list of APIs yet implemented also.

- DataFrame.values
- DataFrame.to_pickle
- DataFrame.memory_usage
- DataFrame.to_xarray

- Series
- Series.values
- Series.to_pickle
- Series.memory_usage
- Series.to_xarray
- Series.array
- Series.duplicated
- Series.real
- Series.nbytes
- Series.__iter__
- Series.ravel

- Index
- Index.values
- Index.memory_usage
- Index.array
- Index.duplicated
- Index.__iter__
- Index.to_list
- Index.tolist

- MultiIndex
- MultiIndex.values
- MultiIndex.memory_usage
- MultiIndex.array
- MultiIndex.duplicated
- MultiIndex.codes
- MultiIndex.levels
- MultiIndex.__iter__
- MultiIndex.to_list
- MultiIndex.tolist