Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add some more common Nodes #503

Open
NickCrews opened this issue Nov 26, 2023 · 1 comment
Open

ENH: add some more common Nodes #503

NickCrews opened this issue Nov 26, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@NickCrews
Copy link
Contributor

Writing pandas dataframes to disk seems really common. I am going to have to write my own DFNode class for this. Would you like me to generalize it enough and then contribute it here?

I would imagine something like

class DFNode:

    def __init__(self, path: str | ..., kind: Literal["csv", "parquet", ...], load_kwargs: dict[str, Any = {}, save_kwargs: dict[str, Any]):
        self.path = path
        self.kind = kind
        self.load_kwargs = load_kwargs
        self.save_kwargs = save_kwargs
    
    def save(self, value: Any) -> None:
        saver = getattr(value, f"to_{self.kind}")
        saver(self.path, **self.save_kwargs)

    def load(self, is_product: bool) -> Any:
        if is_product:
            return self
        loader = getattr(pd,  f"read_{self.kind}")
        return loader(self.path, **self.load_kwargs)
@NickCrews NickCrews added the enhancement New feature or request label Nov 26, 2023
@tobiasraabe
Copy link
Member

Yes, please. I intended to build a collection of nodes that people regularly use with the help of users. If it grows too big, it should probably be split into a separate package, but for now, we can implement them here in a separate module like nodes_extension.py and import necessary packages with import_optional_dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants