Get typing goodies

[gab23r]

I really like the idea to have Generic polars dataframes. But for me the really use case would be to overload polars methods to take advantages of this. For instance, df.rows(named=True) would return a list of typeddict.

I have prototyped something here: https://github.com/gab23r/polaroids (I got the inspiration from this repo).

[baggiponte]

Oh that looks dope! @FBruzzesi made https://github.com/FBruzzesi/anyschema to play with conversion to-and-from different kind of schemas. I sense we can make something nice here!

(By the way, polars-validator is kind of a misnomer. What I am looking for is a way to have Generic frames, so that I can annotate my functions in a meaningful way, and only then perform basic validation. But what I'm after is static analysis. For validation, I like the design behind https://github.com/posit-dev/pointblank)

[gab23r]

Yes I have seen it! Really interesting, something like a narwhal validator could be useful! 😄
I have well understood that the key point is the typing and this is the hard part, as everything has to be static. This is why a TypedDict based system is better than a BaseModel one IMO.

[baggiponte]

Ciao @gab23r , finally have time to look back at this. Have you seen https://github.com/Quantco/dataframely?

I really like the idea to have Generic polars dataframes. But for me the really use case would be to overload polars methods to take advantages of this. For instance, df.rows(named=True) would return a list of typeddict.

I think this could be nice! What are the methods that you'd like to overload? We can start tracking some sub-issues for those.

[gab23r]

As you saw it, I've seen it 😁
it looks great, it it not based on TypedDict, so I don't think we can get a typeddict from a .rows for example. But I see, that they have a mypy plugin for that which is already good!

[baggiponte]

it it not based on TypedDict, so I don't think we can get a typeddict from a .rows for example.

This means that, when you to .rows(), you need to cast everything into the type? I think it's just a type checking thing though, aren't typed dicts just plain dicts at runtime?

[gab23r]

Yes, rows will give you a classical dict ( no typing, no tab completion in the keys of the dict ) and, afaict, you cannot build TypedDict class programmatically. So you can't get TypedDict objects if you don't write explicitly the TypedDict class.