proposal: spec: make imported symbols predictable

[rogpeppe]

Currently when a package is imported, it's not possible to tell what package name it uses without going to the package itself to see what name it uses.

This can make it slow to analyse code (for example to jump to a symbol definition). For example, say we want to find the definition of the symbol X in the expression m.X. If m isn't defined in the local package, it's necessary to obtain a copy of every imported package's code to see if it's defined by any of them. For large packages, that can take a long time (many seconds). Even with caching proxies, it can still take a long time if the set of dependencies has recently changed.

There's also the issue of cognitive overhead: just seeing an import statement is not sufficient to know what symbols that import statement brings into scope. For large programs, that overhead can be significant, and is worse when one or more of the dependencies are no longer available - the reader of the code is forced to guess what symbols are imported.

This issue is even worse when "." imports are used to bring all a package's exported symbols into local scope. Even though dot-imports are frowned upon, there is still no universal consensus that they should be avoided.

The goimports tool already implicitly acknowledges this issue by adding an explicit package name when the symbol isn't clear from the import path. Although this helps, this doesn't help in the general case, because tools cannot assume that goimports has been run.

I propose the following:

• dot imports should be disallowed
• the imported symbol for a given import should be made predictable

This would mean that we can always look at any symbol in a piece of Go source code with local context only and know definitively whether it is defined in the local package or not, and if not, exactly which import path would need to be investigated to find the symbol.

Dot imports

The official guidelines suggest that a dot import should only be used "to let the file pretend to be part of package foo even though it is not". This is a stylistic choice and strictly unnecessary. The tests can still use package-qualified names with only minor inconvenience (the same inconvenience that any external user will see).

Other than that, I believe the most common use is to make the imported package feel "close to a DSL". This seems to be actively opposed to the aims of Go, as it makes the programs much harder to read.

Predictable imported symbols for imports

The simplest possibility here would be to require an import symbol for every import path. As many (most?) people use goimports to manage their import statements, this might not be too onerous a requirement.

Another approach would be to state that when lacking an explicit import package name, the package name must match a name derived from the import path. Possible rules might be:

• split the import path on /; the expected name is the last element.
• choose the longest valid Go identifier from the end of the import path.
• whatever rule goimports currently uses to determine whether it should add an explicit package name.

This means that we could carry on importing "fmt" without redundantly specifying fmt "fmt", but has the disadvantage that nothing else in the language specification talks about what's inside an import path.

[mvdan]

I think the cognitive overhead is a very important point. Code will be read many more times than written, and Go is already optimised to be easy to read before being easy to write.

For example, most of the points I've seen in favor of dot imports is that they simplify writing quick programs, or swapping imports without needing to rewrite much of the program. I think the solution to that should be tooling, not adding complexity to the language.

[Merovius]

I like this, as it codifies existing best practices. In general, I'd prefer to derive the identifier from the import-path (i.e. I don't like the "require an import symbol for every import" solution). There is one complication, in that I still think that the identifier should - by default - be the package name. But the package name is specified in the importee, and there is no way to tell from a package alone what name it's imported as.

So maybe this change should also talk about canonical import paths - maybe making them required instead of optional? Maybe getting rid of the "package name" and instead have the package-clause refer to the canonical import path? Just spit-balling. But I do think this proposal only talks about half of the equation and should say something about the role of package names and the importee too.

[Merovius]

Oh, as a corollary, BTW: This could also be a tooling-enforced solution (to address the "the spec doesn't care about import-paths" part) in that the Go tool could refuse to work if a package name doesn't follow the requirements set out - exactly like we do with canonical import paths, where it's the go tool that will refuse to work if you choose the wrong import path.

[rogpeppe]

There is one complication, in that I still think that the identifier should - by default - be the package name. But the package name is specified in the importee, and there is no way to tell from a package alone what name it's imported as.

That's why the proposal says "when lacking an explicit import package name, the package name must match a name derived from the import path". If the package name declared in the package is different (and there's no local name specified in the import), it would be an error.

So it's entirely possible to have a package with name x importable as import path example.com/y, but all importers would be forced to add an explicit alias for it.

[Merovius]

Ah, sorry, read over that.

So it's entirely possible to have a package with name x importable as import path example.com/y, but all importers would be forced to add an explicit alias for it.

Hm, IMO that's not great. It means goimports at least would still need to walk/parse (at least in part) the entire workspace to find an import. Which AIUI is one of the main reasons goimports (and by extension, saving, if you run it as a hook) is slow. If we're touching the rules relating import paths to package names, it seems like a good idea to just go ahead and codify the best practice of "last component of import path == package name" once and for all.

But either way, I think this is a good idea :)

[rogpeppe]

Hm, IMO that's not great. It means goimports at least would still need to walk/parse (at least in part) the entire workspace to find an import. Which AIUI is one of the main reasons goimports (and by extension, saving, if you run it as a hook) is slow. If we're touching the rules relating import paths to package names, it seems like a good idea to just go ahead and codify the best practice of "last component of import path == package name" once and for all.

I think that's going a step too far. It would mean I can't copy a package verbatim to another location and use it without changing the package name it declares. Also, what about versioned import paths? The rule as proposed would allow you to import "foo/bar/v2" as bar, but it would not force you to declare the package name as v2. I don't think we should codify knowledge of versioning syntax into the language standard.

[deanveloper]

I've found dot-imports to be useful when writing tests. For instance:

package mypack_test

import . "github.com/deanveloper/mypack"

// ...

I wouldn't really mind the dot-import being removed. I was just adding my particular use case for them.

[bcmills]

FWIW, the algorithm that I'm using in the current draft of goforward is: if the import path ends in /vN for some integer N, use the second-to-last component; otherwise, use the last component.

(Code here.)

CC @heschik (see #28428)

[deanveloper]

What about things like gopkg URLs (gopkg.in/yaml.v1)?

I really like this proposal, just making sure everything is covered. Would yaml be chosen because it's the longest valid Go identifier?

[heschi]

I don't have strong feelings on the proposal itself but I can add some commentary about goimports, since I've been working on it a lot lately and plan to make changes in this area soon.

Per #28428, goimports is going to change to add an explicit name to any import of a package whose doesn't "obviously" match its import path. The algorithm I'm using is the same one Bryan proposed above. And yes, this means that gopkg.in packages have the name "yaml" added to their imports.

With that done, goimports won't need to scan the workspace for a file whose imports are satisfied. It's not perfect, because without this proposal, nothing prevents someone from doing:

import "example.com/foo" // package bar
import "example.com/bar" // package foo

which will confuse goimports into thinking everything's fine. (The only way to avoid the confusion would be to load everything even if the file looks okay, which is exactly what we don't want.) But that's very contrived.