feat(deckgl): add support for OpenStreetMap as our new default and make "tile-providers" more configurable #33603

plavacquery · 2025-05-28T11:07:23Z

SUMMARY

This PR introduces support for OpenStreetMap as the tile provider behind deckgl charts. It also changes the default tile-provider to be OpenStreetMap, since Mapbox isn't as open and requires configuration + an API key. This PR makes tile-providers configurable, so it's possible for those who have an API key and prefer Mapbox to match the default as of before-this-PR.

Changing defaults here is a "breaking change" in the sense that the default is changing, but you can reproduce the previous behavior with a bit of documented configuration.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

In config.py set the following

DECKGL_BASE_MAP = [
     ['tile://https://c.tile.openstreetmap.org/{z}/{x}/{y}.png', 'OpenStreetMap' ],
]

ENABLE_CORS = True
CORS_OPTIONS: dict[Any, Any] = {
    "origins": [
        "https://c.tile.openstreetmap.org",
    ]
}

And in TALISMAN_DEV_CONF
add "https://c.tile.openstreetmap.org" to 'connect-src' array

ADDITIONAL INFORMATION

Has associated issue :Add tile layer (XYZ) support in deck.gl maps #27475
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

korbit-ai · 2025-05-28T11:07:29Z

Korbit doesn't automatically review large (3000+ lines changed) pull requests such as this one. If you want me to review anyway, use /korbit-review.

plavacquery · 2025-05-28T11:08:26Z

Coming from MR : #32867 (Inadvertently closed by failed rebase)

@mistercrunch I have implemented what you asking for about OSM map with deckgl.
Map examples are also set with OSM.

github-actions · 2025-05-30T18:45:15Z

@mistercrunch Processing your ephemeral environment request here. Action: up. More information on how to use or configure ephemeral environments

mistercrunch · 2025-05-30T18:45:50Z

Nice! firing up CI and an ephemeral env. Really excited that deckgl-powered visualization will now work out of the [map]box!!!

mistercrunch · 2025-05-30T18:46:38Z

UPDATING.md

@@ -47,6 +47,7 @@ There's a migration added that can potentially affect a significant number of ex
 - [30284](https://github.com/apache/superset/pull/30284) Deprecated GLOBAL_ASYNC_QUERIES_REDIS_CONFIG in favor of the new GLOBAL_ASYNC_QUERIES_CACHE_BACKEND configuration. To leverage Redis Sentinel, set CACHE_TYPE to RedisSentinelCache, or use RedisCache for standalone Redis
 - [31961](https://github.com/apache/superset/pull/31961) Upgraded React from version 16.13.1 to 17.0.2. If you are using custom frontend extensions or plugins, you may need to update them to be compatible with React 17.
 - [31260](https://github.com/apache/superset/pull/31260) Docker images now use `uv pip install` instead of `pip install` to manage the python envrionment. Most docker-based deployments will be affected, whether you derive one of the published images, or have custom bootstrap script that install python libraries (drivers)
+- [33603](https://github.com/apache/superset/pull/33603) OpenStreetView has been promoted as the new default for Deck.gl visualization since it can be enabled by default without requiring an API key. If you have Mapbox set up and want to disable OpenStreeView in your environment, please follow the steps documented here [https://superset.apache.org/docs/configuration/map-tiles].


Nice! A+ for change-management here!

codecov · 2025-05-30T18:50:53Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.95%. Comparing base (76d897e) to head (b47fb9f).
Report is 1959 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #33603       +/-   ##
===========================================
+ Coverage   60.48%   72.95%   +12.47%     
===========================================
  Files        1931      558     -1373     
  Lines       76236    40283    -35953     
  Branches     8568     4222     -4346     
===========================================
- Hits        46114    29390    -16724     
+ Misses      28017     9802    -18215     
+ Partials     2105     1091     -1014

Flag	Coverage Δ
hive	`47.22% <100.00%> (-1.94%)`	⬇️
javascript	`?`
mysql	`71.94% <100.00%> (?)`
postgres	`72.01% <100.00%> (?)`
presto	`50.97% <100.00%> (-2.83%)`	⬇️
python	`72.92% <100.00%> (+9.41%)`	⬆️
sqlite	`71.51% <100.00%> (?)`
unit	`100.00% <ø> (+42.36%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-05-30T18:54:27Z

@mistercrunch Ephemeral environment spinning up at http://44.248.152.230:8080. Credentials are 'admin'/'admin'. Please allow several minutes for bootstrapping and startup.

mistercrunch · 2025-05-30T21:38:02Z

The ephemeral environment doesn't seem to load openstreetmap out of the box (?)

mistercrunch · 2025-05-30T21:39:10Z

github-actions · 2025-06-04T23:19:31Z

@mistercrunch Processing your ephemeral environment request here. Action: up. More information on how to use or configure ephemeral environments

github-actions · 2025-06-04T23:36:32Z

@ Ephemeral environment creation failed. Please check the Actions logs for details.

mistercrunch · 2025-06-06T21:35:57Z

requirements/base.txt

@@ -119,6 +119,8 @@ flask-caching==2.3.1
    # via apache-superset (pyproject.toml)
 flask-compress==1.17
    # via apache-superset (pyproject.toml)
+flask-cors==4.0.2


oh, wondering if you added this manually? This is not self-evident, but this file is supposed to be compiled by uv pip compile via the script at https://github.com/apache/superset/blob/master/scripts/uv-pip-compile.sh . I think the proper way is to alter https://github.com/apache/superset/blob/master/requirements/base.in#L19 to add a line referencing to .[cors], and then running the script...

I'm trying to clarify how it should be done here -> #33714

In you case I think adding a line for .[cors] in base.in makes most sense. Either that or promoting the one dep in cors to the core set of dependencies in pyproject.toml

github-actions · 2025-06-12T22:40:23Z

@mistercrunch Processing your ephemeral environment request here. Action: up. More information on how to use or configure ephemeral environments

github-actions · 2025-06-12T22:51:09Z

@mistercrunch Ephemeral environment spinning up at http://44.246.245.101:8080. Credentials are 'admin'/'admin'. Please allow several minutes for bootstrapping and startup.

mistercrunch · 2025-06-12T23:15:41Z

NICE!

mistercrunch · 2025-06-12T23:18:26Z

superset-frontend/packages/superset-ui-core/src/validator/validateMapboxStylesUrl.ts

 * @param v
 */
 export default function validateMapboxStylesUrl(v: unknown) {
  if (
    typeof v === 'string' &&
    v.trim().length > 0 &&
-    v.trim().startsWith('mapbox://styles/')
+    (v.trim().startsWith('mapbox://styles/') ||


How about trim() once above the if block?

mistercrunch · 2025-06-12T23:19:17Z

superset-frontend/plugins/legacy-preset-chart-deckgl/package.json

-    "@deck.gl/core": "^9.0.37",
-    "@deck.gl/layers": "^9.0.38",
-    "@deck.gl/react": "^9.1.4",
+    "@deck.gl/aggregation-layers": "^9.1.9",


nice side-win to upgrade to a newer deck.gl!

wow btw looks like this ends up affecting 20k LoC in pacakge-lock.json (!?) curious on doing a sanity check on what's happening in there.... GitHub won't let me expand and don't have time to pull the branch to diff it, but would be great if you can validate/explain why changing a few lines here results in so much package-lock changes (?)

Indeed i pushed package-lock.json after making an "npm i --force", should be fix now

looks much better now!

mistercrunch · 2025-06-12T23:24:55Z

superset-frontend/packages/superset-ui-core/src/validator/validateMapboxStylesUrl.ts

-    v.trim().startsWith('mapbox://styles/')
+    (v.trim().startsWith('mapbox://styles/') ||
+      v.trim().startsWith('tile://http') ||
+      v.trim().includes('osm') ||


this seems a bit dangerous, seems startswith https://tile.osm or https://tile.openstreetmap might be safer. There's probably a fancy one-liner-type expression like allowedPrefix.some(s => trimmed.startswith(s)))

mistercrunch

Added a few comments but super bullish on pushing this through! Amazing work!

QQ, would it be easy to add a few more grayscale, perhaps a dark and light more in there? Curious what tiles are provided as part of OSM and if it's easy to match the kind of diversity that we have on the Mapbox side of the house...

imkonsowa · 2025-06-17T11:58:00Z

Thank you guys or the great work here, are you going to release this soon? I have the same need of configuring map tiles server and it would be great to have it with superset!

mistercrunch · 2025-06-17T16:32:23Z

are you going to release this soon?

Happy to merge once the few minor comments are addressed. Likely to first hit in 6.0, which we'll cut and stabilize as soon as 5.0 is released, and the large Theming-related branch is merged.

plavacquery · 2025-06-18T20:56:45Z

Added a few comments but super bullish on pushing this through! Amazing work!

QQ, would it be easy to add a few more grayscale, perhaps a dark and light more in there? Curious what tiles are provided as part of OSM and if it's easy to match the kind of diversity that we have on the Mapbox side of the house...

I tried to find new ones but unfortunately there are only availables when connecting on their website

github-actions · 2025-06-23T20:05:34Z

@mistercrunch Processing your ephemeral environment request here. Action: up. More information on how to use or configure ephemeral environments

mistercrunch

LGTM! I fired up a fresh ephemeral env to make sure things look good and OSM loads out-of-the-box. Will do some basic checks there before merging.

Amazing contribution btw ❤️ ❤️ ❤️

mistercrunch · 2025-06-23T20:12:18Z

Took the freedom to update the PR title as it'll become the commit message in master as we squash-merge it

mistercrunch · 2025-06-23T20:16:50Z

Almost there! Though it looks like there's a unit test failing here -> https://github.com/apache/superset/actions/runs/15833515890/job/44633495868?pr=33603

github-actions · 2025-06-23T20:17:18Z

@mistercrunch Ephemeral environment spinning up at http://35.89.168.235:8080. Credentials are 'admin'/'admin'. Please allow several minutes for bootstrapping and startup.

mistercrunch · 2025-06-23T20:26:16Z

Works! Interestingly my localstorage is in dark mode, which doesn't look perfect, but still a great step forward versus not-tiles-at-all.

mistercrunch · 2025-06-23T20:32:08Z

I tried to find new ones but unfortunately there are only availables when connecting on their website

GPT seems to think there are some free/open options here -> https://chatgpt.com/share/6859b984-7d54-8010-8b86-10d483bc4fa2

Looks a bit rough and out-of-place in dark mode ->

mistercrunch · 2025-06-23T20:36:02Z

For context Theming + dark mode just landed in master, and we may have to think how we'll support visualization theming to align with this across the board. Seems most core visualization are now able to adapt nicely using alpha transparency, but for something like those deck.gl visualization, we may want to have the chart define options like "Map Style for Light Mode" and another "Map Style for Dark Mode", where the person saving the chart would have the option to specify both, and we'd have a bit of logic checking if the user is in dark mode or not and apply the right tiles based on that.

It's not a requirement to sort this out prior to merging this PR, but would be nice to eventually have that option.

mistercrunch · 2025-06-23T23:56:29Z

About the conflict on package-lock, we get that all the time. Usually you just want to git checkout master superset-frontend/package-lock.json and then re-run npm i, commit and push

pull-request-size bot added the size/L label May 28, 2025

github-actions bot added doc Namespace | Anything related to documentation plugins dependencies:npm packages labels May 28, 2025

dosubot bot added the viz:charts:deck.gl Related to deck.gl charts label May 28, 2025

justinpark requested review from rusackas, betodealmeida and kgabryje May 28, 2025 17:21

mistercrunch added the testenv-up label May 30, 2025

mistercrunch reviewed May 30, 2025

View reviewed changes

plavacquery force-pushed the config-map-tiles branch from 464f56d to fb7bdd0 Compare June 4, 2025 17:55

mistercrunch added testenv-up and removed testenv-up labels Jun 4, 2025

mistercrunch reviewed Jun 6, 2025

View reviewed changes

mistercrunch added testenv-up and removed testenv-up labels Jun 12, 2025

mistercrunch reviewed Jun 12, 2025

View reviewed changes

plavacquery force-pushed the config-map-tiles branch from b47fb9f to cefa7e1 Compare June 18, 2025 20:19

plavacquery added 6 commits June 23, 2025 21:28

fix(rebase): fusion packagelock

3df24f3

docs(maps): add map tiles setting 1

fe59791

fix(lib): add lib flask-cors in base requirements

cd1db20

fix(dependencies): adding cors in base.in

f9fd7d5

fix(comment): validation osm 1

ced3e6f

fix(rebase): packagelock

5b9fce0

plavacquery force-pushed the config-map-tiles branch from 779a2ea to 5b9fce0 Compare June 23, 2025 19:39

mistercrunch added testenv-up and removed testenv-up labels Jun 23, 2025

mistercrunch approved these changes Jun 23, 2025

View reviewed changes

mistercrunch changed the title ~~feat(deckgl): make map tiles configurable~~ feat(deckgl): add support for OpenStreetMap as our new default and make "tile-providers" more configurable Jun 23, 2025

mistercrunch added the risk:breaking-change Issues or PRs that will introduce breaking changes label Jun 23, 2025

fix(test): mapbox msg

098e41f

feat(deckgl): add support for OpenStreetMap as our new default and make "tile-providers" more configurable #33603

Are you sure you want to change the base?

feat(deckgl): add support for OpenStreetMap as our new default and make "tile-providers" more configurable #33603

Conversation

plavacquery commented May 28, 2025 • edited by mistercrunch Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

korbit-ai bot commented May 28, 2025

Uh oh!

plavacquery commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 30, 2025

Uh oh!

mistercrunch commented May 30, 2025

Uh oh!

mistercrunch May 30, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented May 30, 2025

Uh oh!

mistercrunch commented May 30, 2025

Uh oh!

mistercrunch commented May 30, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

github-actions bot commented Jun 4, 2025

Uh oh!

mistercrunch Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mistercrunch Jun 6, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 12, 2025

Uh oh!

github-actions bot commented Jun 12, 2025

Uh oh!

mistercrunch commented Jun 12, 2025

Uh oh!

mistercrunch Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

mistercrunch Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

mistercrunch Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

plavacquery Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mistercrunch Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

mistercrunch Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mistercrunch left a comment

Choose a reason for hiding this comment

Uh oh!

imkonsowa commented Jun 17, 2025

Uh oh!

mistercrunch commented Jun 17, 2025

Uh oh!

plavacquery commented Jun 18, 2025

Uh oh!

github-actions bot commented Jun 23, 2025

plavacquery commented May 28, 2025 •

edited by mistercrunch

Loading

plavacquery commented May 28, 2025 •

edited

Loading

codecov bot commented May 30, 2025 •

edited

Loading

mistercrunch Jun 6, 2025 •

edited

Loading

plavacquery Jun 18, 2025 •

edited

Loading

mistercrunch Jun 12, 2025 •

edited

Loading

mistercrunch commented Jun 23, 2025 •

edited

Loading