Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterate over TypedColumns with evidence #739

Open
gdahia opened this issue Aug 14, 2023 · 0 comments
Open

Iterate over TypedColumns with evidence #739

gdahia opened this issue Aug 14, 2023 · 0 comments

Comments

@gdahia
Copy link

gdahia commented Aug 14, 2023

I was wondering if there was a way to iterate over the columns as Symbols with the appropriate evidence of their existence in the typed dataset, and perhaps some type tagging to ensure the operations for that symbol are type compatible.

My concrete example is the following: I have a dataset for which some columns are String, and want to replace those columns by translated version of their contents. Some (pseudo)code illustrating what I wanted to achieve is

val ds = TypedDataset(...)
ds.columns.foldLeft(ds) { (ds, typedColumn) =>
  typedColumn match {
    case typedColumn: TypedColumn[String] => ds.withColumnReplaced(typedColumn.symbol, translate(typedColumn))
    case _ => ds
  }
}

Is that already possible?

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant