Why are libraries bundled instead of using submodules?

[GravisZro]

I've noticed that the code of libraries have been copied into libgit2 instead of being used in place as submodules. The problem with this is that you lose the ability to easily update the code utilized and it further complicates licensing. This raises the question: why are libraries bundled instead of using submodules?

[ethomson]

Submodules aren't actually a very appropriate use here. Some of our dependencies — notably, xdiff — doesn't actually have its own upstream repository. https://github.com/git/git is its upstream repository, and I don't want to make git a submodule of libgit2.

Even the ones that do live in their own directory have changes to fit into our build system. Even ntlmclient, which is basically written for libgit2, doesn't include cleanly. And bringing in some other repo as a submodule and then hacking at it to change it is going to be deeply unsatisfying.

I'm not sure what this has to do with licensing?

[GravisZro]

Some of our dependencies — notably, xdiff — doesn't actually have its own upstream repository

True... but you also made your own upstream repository, a feat that can be replicated.

https://github.com/git/git is its upstream repository, and I don't want to make git a submodule of libgit2.

Yes, some copied code isn't from a library which makes it's incorporation understandable.

Even the ones that do live in their own directory have changes to fit into our build system.

The "proper way" of doing this is to fork the upstream repository, make a libgit2 specific branch, apply your changes, and then add that branch as a submodule. This allows you to pull from upstream and rebase the libgit2 specific branch. The result is the library code is updated while the libgit2 specific changes are preserved as well as preserving commit history. Keeping up to date is important in the context of bug fixes and security.

I'm not sure what this has to do with licensing?

When bundling code you have to incorporate the foreign code's license into the license file which is why the license for libgit2 is actually a list of 17 different licenses.

In my experience, bundling large amounts of code is generally (not always) the worse option for a project (especially complex ones) so I'm merely investigating the reasoning behind it. Thus far, I still believe that submodules are the appropriate choice here.

[GravisZro]

If you're wondering what the "proper way" looks like, check out the clar submodule from an example branch: https://github.com/GravisZro/libgit2/tree/unbundle/tests

Note that the clar repo contains the commits from both the original and the libgit2 repository and the git log retains the correct timestamps.

[ethomson]

When bundling code you have to incorporate the foreign code's license into the license file which is why the license for libgit2 is actually a list of 17 different licenses.

Why do you believe that you would need to incorporate the license if it's bundled in-tree but it would be different if it would be bundled as submodules? How does the resultant binary change and how does the license affect that?

Is this opinion driven by discussions with your IP counsel?

In any case, I'm unlikely to start using submodules for our dependencies. The ergonomics are not good for the libgit2 developers or our users.

[GravisZro]

Why do you believe that you would need to incorporate the license if it's bundled in-tree but it would be different if it would be bundled as submodules?

I do not. What it does do is simplify the licensing auditing because each repository has it's own license file. Presently, identifying the license of a portion of code is extremely painful. This also makes it easily modular so that portions can be replaced if desired.

How does the resultant binary change and how does the license affect that?

The binary and code licensing would be identical. What this impacts is the source code and makes it easier to developers to understand how the code is licensed without trawling through a huge file of licenses. Right now it's trivial to inadvertently make a commit can alter code that will be licensed differently.

In any case, I'm unlikely to start using submodules for our dependencies.

That is disappointing.

The ergonomics are not good for the libgit2 developers or our users.

I'm not sure why you think that as the actual on disk tree structure would be identical. Honestly, I think the security implications should be more than enough to switch to submodules.