go.dev: support packages with unknown/unsupported licenses

[eliasnaur]

There has been work on this, but @mvdan suggested I create an issue for it for tracking.

The motivating example is the dual-licensed (MIT/UNLICENSE) gioui.org packages. See https://pkg.go.dev/gioui.org/op. The problem is that go.dev refuses to display certain meta-data if the license is not detected or in a list of supported licenses. In particular, the blocking of "Doc" and "Overview" sections make the pkg.go.dev listing almost useless.

Compare godoc.org which, while inferior to pkg.go.dev in other ways, has no problem displaying documentation regardless of licenses.

The Go team seems to have a manual process of vetting packages such as gioui.org. However, that process is slow for all parties and might not cover every weird license out there.

@rogpeppe because pkg.go.dev also blocks some of his packages.

[julieqiu]

We discussed this on the golang-tools call - but posting here for posterity.

At the moment, we can only post metadata about a package/module (imports, importers, published date, etc.) when it does not have a recognized license per the requirements at https://pkg.go.dev/license-policy.

If any package authors are having issues, feel free to email go-discovery-feedback@google.com for manual review consideration by our legal team.

[mvdan]

I think Julie misclicked - from her comment, I don't understand why we should close this issue. It seems useful to leave it open, so that other module authors having similar issues can subscribe for updates.

[julieqiu]

I closed this issue because we don't intend to support unknown/unsupported licenses in the foreseeable future.

The most up-to-date information on our license policy will be at https://pkg.go.dev/license-policy. Authors can feel free to file an issue, or email go-discovery-feedback@google.com for any questions about their specific package.

Any major updates will be posted on the tracking issue for pkg.go.dev, so folks should subscribe to that issue for project updates.

[eliasnaur]

Would you consider alternatives? It seems to me the issue that go.dev needs permission to a very limited use of the copyrighted work, namely displaying the textual documentation embedded in the repository. For example, how about a file that clearly states go.dev (or source code explorers in general) permission to display the package contents? Say, <reporoot>/.allow-sourceexplorers.

[eliasnaur]

FWIW, other source explorers such as https://docs.rs/ doesn't seem to be affected by licensing issues either.

[julieqiu]

We (and the proxy team) considered that option and decided that it wasn't viable. We would rather not solve this problem with a "just for us" solution, because it doesn't encourage package authors to fix their licenses for the broader open source ecosystem if there are issues, and other users have concerns about a package or module's licenses.

/cc @jba @findleyr @katiehockman @heschik @hana

[eliasnaur]

a "just for us" solution

I deliberately named the file without reference to go.dev. I assumed that if Google has this legal issue with displaying source code meta-data, other similar explorers have it as well. Otherwise, what's special about Google?

package authors to fix their licenses for the broader open source ecosystem if there are issues, and other users have concerns about a package or module's licenses.

I'm sorry, but I can't see what I can do to "fix my license" for the gioui.org modules. If anything, go.dev is being overly restrictive in not accepting dual licenses as OR, rather than AND (gioui.org is dual licensed MIT OR UNLICENSE and MIT is supported by go.dev).

Secondly, I don't think it proper for a source code explorer to dictate the "proper" licenses for Go package.

Of course, please disregard my comments if you consider free, manual legal review by Google an acceptable solution for every package that needs it. If so, I still think there should be an automatic, faster (and cheaper for Google?) way to smooth the process; hence my .allow-sourceexplorers strawman.

[julieqiu]

gioui.org will be fixed on our next deploy, so there isn't any further action needed on your part.

I will have to defer to our legal team on their decision for how to the AND vs OR decision for dual licenses because IANAL.

Our current volume of special case issues is low enough that we plan to keep the manual review process for now (rather than building an automatic solution). We can always revisit this strategy as other specific issues come up.

[mewmew]

gioui.org will be fixed on our next deploy, so there isn't any further action needed on your part.

Good that gioui.org is now supported by pkg.go.dev. However, what about other packages that are dual-licensed, that are put in the public domain (through UNLICENSED) or otherwise blocked by pkg.go.dev?

There are currently 2,485 Go repositories released into the public domain (through UNLICENSED). For comparison, the total number of open source Go repositories hosted on GitHub is 232,299, so ~1% are released into the public domain.

To give another comparison, there are more Go projects released into the public domain (through UNLICENSED) than there are Go projects licensed under AGPL3, of which there are currently 2,244. And AGPL3 is recognized by pkg.go.dev.

There are currently 39 Go repositories licensed as Artistic License 2.0, another license recognized by pkg.go.dev.

For a full set of statistics, see below:

• mit: MIT License
  • 112,974 (approved by pkg.go.dev)
• apache-2.0: Apache License 2.0
  • 45,046 (approved by pkg.go.dev)
• gpl-3.0: GNU General Public License v3.0
  • 14,517 (approved by pkg.go.dev)
• bsd-3-clause: BSD 3-Clause "New" or "Revised" License
  • 8,157 (approved by pkg.go.dev)
• bsd-2-clause: BSD 2-Clause "Simplified" License
  • 4,059 (approved by pkg.go.dev)
• gpl-2.0: GNU General Public License v2.0
  • 2,780 (approved by pkg.go.dev)
• unlicense: The Unlicense
  • 2,485 (not approved by pkg.go.dev)
• mpl-2.0: Mozilla Public License 2.0
  • 2,445 (approved by pkg.go.dev)
• agpl-3.0: GNU Affero General Public License v3.0
  • 2,244 (approved by pkg.go.dev)
• lgpl-3.0: GNU Lesser General Public License v3.0
  • 1,858 (approved by pkg.go.dev)
• isc: ISC License
  • 1,433 (approved by pkg.go.dev)
• wtfpl: Do What The F*ck You Want To Public License
  • 552 (not approved by pkg.go.dev)
• cc0-1.0: Creative Commons Zero v1.0 Universal
  • 231 (not approved by pkg.go.dev)
• lgpl-2.1: GNU Lesser General Public License v2.1
  • 167 (approved by pkg.go.dev)
• zlib: zlib License
  • 162 (approved by pkg.go.dev)
• epl-1.0: Eclipse Public License 1.0
  • 110 (not approved by pkg.go.dev)
• epl-2.0: Eclipse Public License 2.0
  • 46 (not approved by pkg.go.dev)
• bsl-1.0: Boost Software License 1.0
  • 41 (approved by pkg.go.dev)
• artistic-2.0: Artistic License 2.0
  • 39 (approved by pkg.go.dev)
• cc-by-sa-4.0: Creative Commons Attribution Share Alike 4.0 International
  • 27 (not approved by pkg.go.dev)
• bsd-3-clause-clear: BSD 3-Clause Clear License
  • 18 (not approved by pkg.go.dev)
• osl-3.0: Open Software License 3.0
  • 14 (not approved by pkg.go.dev)
• eupl-1.2: European Union Public License 1.2
  • 14 (not approved by pkg.go.dev)
• cc-by-4.0: Creative Commons Attribution 4.0 International
  • 12 (not approved by pkg.go.dev)
• 0bsd: BSD Zero Clause License
  • 6 (not approved by pkg.go.dev)
• afl-3.0: Academic Free License v3.0
  • 5 (not approved by pkg.go.dev)
• ms-pl: Microsoft Public License
  • 3 (not approved by pkg.go.dev)
• eupl-1.1: European Union Public License 1.1
  • 2 (not approved by pkg.go.dev)
• bsd-4-clause: BSD 4-Clause "Original" or "Old" License
  • 2 (not approved by pkg.go.dev)
• odbl-1.0: ODC Open Database License v1.0
  • 1 (not approved by pkg.go.dev)
• upl-1.0: Universal Permissive License v1.0
  • 0 (not approved by pkg.go.dev)
• postgresql: PostgreSQL License
  • 0 (not approved by pkg.go.dev)
• ofl-1.1: SIL Open Font License 1.1
  • 0 (not approved by pkg.go.dev)
• ncsa: University of Illinois/NCSA Open Source License
  • 0 (not approved by pkg.go.dev)
• ms-rl: Microsoft Reciprocal License
  • 0 (not approved by pkg.go.dev)
• lppl-1.3c: LaTeX Project Public License v1.3c
  • 0 (not approved by pkg.go.dev)
• ecl-2.0: Educational Community License v2.0
  • 0 (not approved by pkg.go.dev)
• cecill-2.1: CeCILL Free Software License Agreement v2.1
  • 0 (not approved by pkg.go.dev)

---

Total number of open source Go repositories on GitHub:

• 293,872

Total number of open source Go repositories with license recognized GitHub search:

• 199,450

Total number of open source Go repositories with license approved by pkg.go.dev:

• 195,922

Total number of open source Go repositories with license not approved by pkg.go.dev:

• 3,528 (out of which 2,485 are projects in the public domain with UNLICENSED)

---

Personally, I think that this helped me gain a better understanding of the license landscape of the Go ecosystem. Prior to this quick evaluation, I felt that the list of licenses was too restrictive. However, given that it is possible to dual-license projects (although it will require manual effort to get them on pkg.go.dev), I think that the list of licenses is good enough as is.

I would very much like to see a public domain option added to the list, as it is the license used by most Go projects which is not yet supported by pkg.go.dev.

With kind regards,
Robin Eklind

--

Edit: I do wonder though, what those 94,422 (32%) of the projects without a license recognized by GitHub search are.

[jba]

Thanks for that analysis. Very illuminating.

We appreciate what motivates people to reach for a public-domain style license, and in our own analysis, we've also identified that as a key issue. We'll try to have an update on that topic soon.