Discussion: Consider adding a debug mode to show all data in tiles

[ebrelsford]

Maputnik uses a library to do this but the opinion of the team is that it isn't good enough:

We should consider

• Is there a way to make the library Maputnik uses create a better view?
• Is there another library we could use?
• Should we do this ourselves?
• Distinguishing between features that appear on the map and those filtered out by the style

[almccon]

To clarify, do we mean something specific by "all rendered feature data" as opposed to all feature data in the tiles?

[ebrelsford]

Good point! Updated the name to be more open in that regard.

[almccon]

Here's it's also good to look at what https://stevage.github.io/vector-inspector vector inspector does well.

[aparlato]

I'm doing some thinking about this now and would like to get a better sense of where the inspect mode in Maputnik fails so that we know how we might want this to work or whether something closer to #43 might serve us better in this tool.

I'd also like to check in on the use case here: Do we think we'll use this mostly to compare a style to its own debug style to see how data is filtered or are we thinking we'd compare the debug styles of two separate styles to get a sense of how the data compares?

Distinguishing between features that appear on the map and those filtered out by the style

One thing to consider here is that, when looking at all source layers and all style layers here, it's likely that this view will be overwhelming. In Maputnik, we have the advantage of being able to show filtered vs unfiltered features/data based on a single selected layer. This is something we don't have in this compare tool. And it may result in this tool trying to do too much if we allow selecting a style or source layer.

Another potential issue with using the map style as the basis of our debug style is the number of sources. If we're concerned mostly with seeing two sources next to each other to see how they compare, maybe the right approach is to allow a dropdown option to insert a link to a tile json or pbf similar to the Vector inspector to create a style that isn't connected to a real map style that represents that data.

Lastly, regardless of path forward here, my inclination is to see how far we can get with a more scoped solution like #43 before trying to implement this.

Curious to hear your thoughts @ebrelsford @almccon !

[ebrelsford]

I'd also like to check in on the use case here: Do we think we'll use this mostly to compare a style to its own debug style to see how data is filtered or are we thinking we'd compare the debug styles of two separate styles to get a sense of how the data compares?

I'm also curious about this. I was mostly thinking the former (compare the rendered style to the source data to see what else is available in that source) but I could see the latter being interesting too.

If we think about this tool as part of a replacement for Maptunik (editing happens in a code editor, viewing happens in this tool), then I can see the argument for bringing more of the viewing and analysis features from Maptunik into this tool. I can see how making a separate tool for viewing source data could be cumbersome if your development workflow is:

1. edit in code editor
2. build with build system
3. view in this tool

One thing to consider here is that, when looking at all source layers and all style layers here, it's likely that this view will be overwhelming.

Agreed. And if we were to build a UI for doing layer selection, would that be useful in the tool generally or would it only be useful here? And at that point is this better off in its own tool?

If we're concerned mostly with seeing two sources next to each other to see how they compare, maybe the right approach is to allow a dropdown option to insert a link to a tile json or pbf similar to the Vector inspector to create a style that isn't connected to a real map style that represents that data.

I think this might be useful either way in terms of tile dev. Being able to compare the current tile source with a tile or two of dev tiles would be handy...

Lastly, regardless of path forward here, my inclination is to see how far we can get with a more scoped solution like #43 before trying to implement this.

The sounds good to me, let's start with #43.

[aparlato]

I can see how making a separate tool for viewing source data could be cumbersome if your development workflow is:

1. edit in code editor
2. build with build system
3. view in this tool

Just to throw another idea out here if this is a blocker to creating a separate tool: In client projects, we could make this less cumbersome if we wanted (with a little more work) by threading tools together. If we had a separate tool for this functionality, structured similarly, we might be able to add a custom link between tools right in the client specific repo's HTML file to switch between them without restarting a server. There are likely some issues to work out here and we'd have to force the link to open a new tab so that we don't change the URL that persists state though.

[kelsey-taylor]

I'm doing some thinking about this now and would like to get a better sense of where the inspect mode in Maputnik fails so that we know how we might want this to work or whether something closer to #43 might serve us better in this tool.

What fails for me with the data view mode in Maputnik is that you only see the data for features in whichever layer you have selected. Being able to see everything would be so much more useful in most cases

Distinguishing between features that appear on the map and those filtered out by the style

Mapbox Studio does a really nice job at this that we could emulate

[RossThorn]

Just to weigh in with nothing new, the number one use case to me is just seeing the data in the tiles. It's really helpful to see the data and all properties that we can use when creating styles. The actual "what's rendering" or "what's being filtered out" feature seems like it could be cool too!

Another potential use would be a focused tile view where I can look at the data in just one or a couple tiles and see all the features listed or grouped by tags/sources in a slightly different UI than something listing all of the layers. This aids in helping troubleshoot specific areas if something is appearing buggy in a certain area. So a focused tile view and a focused data view for those specific tiles would be helpful.

What fails for me with the data view mode in Maputnik is that you only see the data for features in whichever layer you have selected. Being able to see everything would be so much more useful in most cases

@kelsey-taylor I thought this was the case as well, but it doesn't appear to be that way for all styles? I'm kinda just finding this out now. I've only really used Maputnik for our stylesheets that we create for clients and it's true that you can only see the data in the layers you have selected. But if I look at the OSM Liberty map style (one of the default selections in Maputnik), I can see everything!

for context, here's the same area above just loading one of our own style jsons. Kelsey is right in that it's blank unless I select a layer, then it only shows that specific layer.

So maybe Maputnik can be a good candidate and we need to figure out why our style jsons don't show every layer when loaded into the editor, and if there's anything we could do on our json generation or could do in our Maputnik fork to better suit tile inspection with this method.

Maybe one or all of you already know why that is 🤷

[kelsey-taylor]

@kelsey-taylor I thought this was the case as well, but it doesn't appear to be that way for all styles?

I was thinking the same thing! I don't remember it being so limiting when I worked on a set of styles with their own tiles last year, but we had a separate fork with some changes that I assumed explained the difference. Not sure what's going on there 👀

[ebrelsford]

I dug into this a little by making the simplest version of one of our styles and it still didn't work. I imagine there's some debugging we could do around this part of Maputnik if we decided that was a possible avenue. Most of the work there is done by mapbox-gl-inspect.

[kateler]

Unlurking to shed some light on this. The difference is whether there is a tile.json file for the source, or if it points to tiles directly. A tile.json contains the usual source info plus a basic schema for the tiles, telling the inspect function things like what layers are present, data types, and what kind of geometry is included.

Here's the OMT file, which is especially detailed. I created tile.json files at Amazon that were simpler and didn't include all possible values, and those worked fine too.

[ebrelsford]

Oh duh, that makes sense. Thanks @kateler!

[dnomadb]

Related: https://github.com/stamen/carto-tools/issues/16

[mulloverit]

Grooming discussion

• Big missing piece from our workflow is having a debug mode
• Maperture seems like a good place to implement this since it is already heavily used
• Other tools already do this, like Maputnik (screenshots above) and Vector Inspector, but it doesn't quite work for our current styles
• Question: do we spend the time to incorporate this into Maperture or do we find another tool that we're happy with using

Maperture or not

• Feels like a natural extension of Maperture
• Pairs nicely with Add layer name to feature popup #157
• Using another tool would be cumbersome because you'd need to c/p URLs around to make sure your view is the same
• Maperture would have compare panes which is highly convenient for viewing
• Development time is a question mark, which makes us hesitant to pick this up
• Tile.json vs a tile endpoint are very different tasks - doing this for a tile endpoint would be a much larger lift
• Would be most useful to see both the rendered data and raw tile data to know what's available to be used
• Note on building this: it is about creating a style on the fly; let's build in a modular fashion
• Feels like we should have a list of layers for client tile endpoints anyway; is creating a tile.json something that we'd want to build into this tooling?
• How many clients do we have that are using a tile.json instead of tile endpoints?
  • AWS does not use tile.json
  • Lyft does not use tile.json
  • Last Mile had a tile.json, but only because Kate created it for them
• Advocating for us to push clients to expose this information to us

User story

As a user of Maperture, I want to be able to understand how tile data correlates to the style that I'm working on. The ability to do this in Maperture would be much less cumbersome than current solutions (which force us to jump between applications) and would speed up our style development work. This is the kind of tooling that is helpful when styling a feature for the first time in order to understand the frequency/density of that feature. This would also help find feature/tagging anomalies in tiles.

Acceptance criteria

• Maperture has a button that enters a viewer into debug mode, where all tileset data is displayed in a generated style
• In debug mode, all data from all source tilesets in the style should be displayed
• This should support both tilejson and tiles endpoint sources
• When the debug view is enabled, tooltip actions stay active and display source information

Without leaving this tool, I can look at the data in a tile, see what features and layers are being used and not.

Client impacts

• Workflow enablement: we'll be able to more quickly iterate on styles. We'll reduce the friction of context switching for jumping between tools
• Client opportunity: articulates opportunities for tileset improvements, which fits into our larger goals for vector tile cartography services