Skip to content

Is there any function that I can turn polars.LazyFrame or polars.Expr into daft.LazyFrame or daft.Expr? #4916

Discussion options

You must be logged in to vote

@baiguoname, to convert to/from polars, it will require a full materialization via .collect in both libraries. Once materialized, you can use arrow to convert between the two at very little cost

# polars to daft
df = pl.scan_parquet('...')
# some df operations
df = df.collect()
daft_df = daft.from_arrow(df.to_arrow())

# daft to polars
df = daft.read_parquet('...')
# some daft operations
df = df.collect()
pl_df = pl.from_arrow(df.to_arrow())

We do not have plans on supporting conversions between polars <-> daft logical plans (LazyFrame) and expressions

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by universalmind303
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants