chore: Extensions architecture POC

[michael-s-molina]

SUMMARY

This branch is for POC purposes and should not be merged to master. It contains the following packages:

extensions
Example extensions used for testing and validating the package dependencies.

superset-frontend/packages/superset-core
Our package with UI components, utility functions, types, and APIs used by extensions.

superset-core
The host APIs used by backend extensions.

superset-cli
CLI commands to build, bundle and hot-rebuild extensions.

superset-extensions build     Builds extension assets.

superset-extensions bundle    Packages the extension into a .supx file.

superset-extensions dev       Provides hot-rebuilding of assets.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

The Dataset references extension displays relevant information for SQL authors in SQL Lab:

• Datasets referenced in a SQL Query: This helps users discover the origin of the data, especially for long SQL queries.
• Owners of each referenced dataset: This is crucial for dealing with access permissions or quickly finding out who to contact in case of problems.
• Latest available partition: This is important for understanding data completeness issues.
• Estimated row count for each dataset: This helps users grasp the implications of joins.

Screen.Recording.2025-06-04.at.10.48.32.mov

TESTING INSTRUCTIONS

1 - Set the ENABLE_EXTENSIONS feature flag to True.

2 - Run npm ci on superset-frontend.

3 - Install the development dependencies by running pip install -r requirements/development.txt. This will pull in the new cli package that contains the superset-extensions command.

4 - Bundle an extension by going into its root folder and executing superset-extensions bundle. This will build the extension and generate a .supx bundle of the extension.

5 - Upload the .supx file to Superset using Swagger and the extensions endpoint.

6 - Go to SQL -> SQL Lab and check the extension.

For development mode:

1 - Execute steps 1, 2 and 3 above.

2 - Configure Superset's LOCAL_EXTENSIONS variable like:

LOCAL_EXTENSIONS = [
    "./extensions/dataset_references",
]

3 - Run in development mode by going into the extension root folder and executing superset-extensions dev. This will trigger hot-rebuilding of your frontend assets. Backend assets are currently reloaded by the default reloading mechanisms of Superset using superset run --reload.

4 - Go to SQL -> SQL Lab and check the extension.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[korbit-ai]

Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment /korbit-review.

Your admin can change your review schedule in the Korbit Console

[codecov]

Codecov Report

Attention: Patch coverage is 1.41343% with 279 lines in your changes missing coverage. Please review.

Project coverage is 72.45%. Comparing base (76d897e) to head (e34133f).
Report is 1937 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/extensions/utils.py	0.00%	105 Missing ⚠️
superset/extensions/api.py	0.00%	68 Missing ⚠️
superset/daos/extension.py	0.00%	36 Missing ⚠️
superset/extensions/models.py	0.00%	29 Missing ⚠️
superset/initialization/__init__.py	5.55%	15 Missing and 2 partials ⚠️
superset/extensions/exceptions.py	0.00%	12 Missing ⚠️
superset/extensions/types.py	0.00%	12 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #31934       +/-   ##
===========================================
+ Coverage   60.48%   72.45%   +11.97%     
===========================================
  Files        1931      564     -1367     
  Lines       76236    40559    -35677     
  Branches     8568     4253     -4315     
===========================================
- Hits        46114    29389    -16725     
+ Misses      28017    10077    -17940     
+ Partials     2105     1093     -1012     

Flag	Coverage Δ
hive	46.89% <1.41%> (-2.26%)	⬇️
javascript	?
mysql	71.45% <1.41%> (?)
postgres	71.51% <1.41%> (?)
presto	50.62% <1.41%> (-3.18%)	⬇️
python	72.42% <1.41%> (+8.91%)	⬆️
sqlite	71.02% <1.41%> (?)
unit	100.00% <ø> (+42.36%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.