-
Notifications
You must be signed in to change notification settings - Fork 253
refactor: make DaftExtension class definition static #4968
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR refactors the DaftExtension
class definition in daft/datatype.py
by moving it from being dynamically created inside the _ensure_registered_super_ext_type()
function to being defined at module level. The change improves code organization and static analysis capabilities without altering the class's functionality.
Previously, the DaftExtension
class was created dynamically within the _ensure_registered_super_ext_type()
function, which made it invisible to static type checkers, IDEs, and other code analysis tools. The new implementation defines the class statically at lines 1178-1199, making it part of the module's permanent structure. The _ensure_registered_super_ext_type()
function (lines 1201-1216) has been simplified to just assign the pre-defined class to _STATIC_DAFT_EXTENSION
and register it with PyArrow when needed.
This change fits well within Daft's Arrow integration architecture, where the framework extends PyArrow's type system for custom data handling. The DaftExtension
serves as the base class for Arrow extension types in Daft, and making it statically available improves the overall maintainability of the Arrow integration layer. The double-checked locking pattern is preserved to ensure thread-safe registration with PyArrow.
Confidence score: 4/5
- This PR is safe to merge with minimal risk as it's a code organization refactoring that maintains identical functionality
- Score reflects straightforward refactoring with no logic changes and preservation of thread safety mechanisms
- No files require special attention as the change is isolated to a single module with clear intent
1 file reviewed, no comments
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4968 +/- ##
==========================================
+ Coverage 78.84% 79.13% +0.29%
==========================================
Files 890 917 +27
Lines 123743 126969 +3226
==========================================
+ Hits 97559 100483 +2924
- Misses 26184 26486 +302
🚀 New features to boost your workflow:
|
@malcolmgreaves When you have time, please take a look at this PR. It aims to solve the UT issue corresponding to #4940. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Jay-ju! I think these changes LGTM. But since these changes exist to make #4940 work, I'd like to actually combine both PRs into one. That way we can make sure that these changes are indeed solving the problem we expect. I want to avoid merging in this PR, updating #4940 with it, then realizing that it didn't fix the problem.
I made #4975 as a test of both this and the other PR's changes. |
This PR's title and description don't actually describe what's happening here. So I will change them to be: title: PR message changes: |
@malcolmgreaves fix it |
Changes Made
daft.datatype._ensure_registered_super_ext_type.<locals>.DaftExtension
vs<class 'daft.datatype.DaftExtension'>
Moves the DaftExtension class definition to the module level. Previously, this class was defined at runtime from within the _ensure_registered_super_ext_type function. This meant that the class definition itself was dynamic, which causes problems for some specific cases of schema handling while writing out to some data formats.
Related Issues
Checklist
docs/mkdocs.yml
navigation