Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List source map implementations #47

Open
littledan opened this issue Jun 13, 2023 · 12 comments
Open

List source map implementations #47

littledan opened this issue Jun 13, 2023 · 12 comments

Comments

@littledan
Copy link
Member

If we're trying to harmonize source map implementations, it would be useful to list all of the implementations we can find, so that we can then compare them. This includes both generators and consumers.

@bomsy
Copy link
Collaborator

bomsy commented Jun 14, 2023

These would also be super helpful for understanding the systems to be tested.

Here is an implementation from mozilla used in firefox
https://github.com/mozilla/source-map

@sjrd
Copy link

sjrd commented Jun 14, 2023

I'll put forward the Scala.js compiler as generator.

Some of our interpretations are:

  • Every source tree carries its source position through transformations (counting the entire pipeline, there are dozens of transformation phases) and eventually reaches every JavaScript AST node.
  • When printing a JS AST node, we emit source map segments that point to its position for all the text it generates, unless overridden by children nodes. So for example, an expression (a + b) attaches the parens, spaces and + sign to the position of the AST node for +, while a and b are attached to the positions of the identifier AST nodes.
  • We only give original names to the definition of identifiers. In let a = 1; a only the first a receives an original name in the source map.
  • Our source files are UTF-8 encoded usually, and strings in Scala are ~UTF-16 (like JS). In terms of output, we always emit ASCII.

In terms of implementation, perhaps one key aspect is that it is specialized for an incremental compilation pipeline. When changing part of a source program, the entire Scala compilation pipeline is incremental end-to-end, and that until re-pretting JS AST nodes and generating source maps. Source maps are actually a pain point here, since they must be regenerated from top to bottom every time. We have an intermediate data structure consisting of Fragments to make that as efficient as we could.

You can see our implementation here:
https://github.com/scala-js/scala-js/blob/v1.13.1/linker/shared/src/main/scala/org/scalajs/linker/backend/javascript/SourceMapWriter.scala
If you're not familiar with Scala, it's worth noting that this file looks nothing like typical Scala code. This is almost C code written with Scala syntax, because of how performance-sensitive it is.

@7rulnik
Copy link

7rulnik commented Jun 14, 2023

There is also a fork of source-map — https://github.com/7rulnik/source-map-js

@mitsuhiko
Copy link
Contributor

This is our Rust library: https://crates.io/crates/sourcemap

The most notable users of this library to the best of my knowledge are Sentry, SWC and Deno.

It's pretty low level, we have higher level wrappers on top of that (js-source-scopes, symbolic, and symbolicator) for consumption.

@jaro-sevcik
Copy link
Contributor

@connor4312
Copy link

There is also https://www.npmjs.com/package/@jridgewell/source-map which is namely used in Terser. It tries to be compatible with Mozilla's source map library, but I have found some variance in the past.

@jkrems
Copy link
Contributor

jkrems commented Jun 14, 2023

Will there be interesting differences between uses of the libraries when it comes to what we'd want to specify? E.g. will all users of mozilla/source-map generate "spec compliant source maps in all ways we care about" as soon as mozilla/source-map has been updated in accordance with a future hardened spec?

If not, it may also be helpful to collect things like postcss which indirectly produce source maps.

@jridgewell
Copy link
Member

I maintain:

@ochameau
Copy link

While trying to revive mozilla original source-map implementation, I went throught all major source map implementations I could easily find online and come up with this summary:
https://docs.google.com/document/d/1cRNI5s9ECI6V7argqw0g7TDp79gNrOZiVYLIRDD9FlA/edit#heading=h.3dsnp8frk1tz
You will see links to sources of each of them and some recap about the specifics of each.

@sokra
Copy link
Collaborator

sokra commented Sep 22, 2023

@legendecas
Copy link
Member

@mischnic
Copy link

https://github.com/parcel-bundler/source-map is used by Parcel and Lightning CSS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests