[DO NOT MERGE] Reference commit for package aliasing

[minosgalanakis]

Description

Simple showcase of how to create an aliased version of a package in python by overriding the __all__ attribute of the module loader.

Unfortunately we cannot re-use the same name and if we would like to contain python files in multiple locations and expose them as a single pacakge we should consider namespace packages

I think it is considerably more complicated, then just using the proposed trick with all imports.

In a script in the main codebase you can do
from mbedtls_dev import * while in the test codebase you could do from mbedtls_test_dev import * and would still expose the modules directly bignum_common so the code won't have to change.

This PR is not to be merged just for discussion purposes.

[gilles-peskine-arm]

I'm not sure I understand: do you mean we can replace from . import other_module by other_module? Or is there more to it?

[minosgalanakis]

My information may be outdated.

When you do from . import module module is not imported from the current directory, but from the top level of the package. . indicated the root of the package.

Overriding __all__ was a surefire way to control what is provided by the package, and then relative imports could work.

To be frank I did not test that specific use-case.

One other note, this globbing functionality does not have to be be here if only one other libary is using it. If we only want to alias this library, from the test scripts, it could go into the mbedtls_test_dev/__init__.py

If though we expect to have libraries in multiple locations, like framework, tests, or ci, then it would be needed at this file, since it is where the actual code base resides in.

e.g

single alias

mbedtls_dev/__init.py__


mbedtls_test_dev/__init.py__
  list = mbedtls_dev.glob()
  from mbedtls_dev import list 

Glob once and re-use

mbedtls_dev/__init.py__
  glob()


mbedtls_test_dev/__init.py__
  from mbedtls_dev import *

mbedtls_framework_dev/__init.py__
  from mbedtls_dev import *

[gilles-peskine-arm]

Unfortunately we cannot re-use the same name and if we would like to contain python files in multiple locations and expose them as a single pacakge we should consider namespace packages

I think it is considerably more complicated, then just using the proposed trick with all imports.

What's complicated about namespace packages? If I'm reading the PEP right, they're available natively since Python 3.3, and we currently require ≥3.5 for our CI and ≥3.8 as the documented minimum for Mbed TLS 3.6 LTS. And as far as I can tell, all they require is a bit of ugliness in __init__.py files, which isn't worse than our current ugliness.

[minosgalanakis]

Complicated as if people wil need to understand it since it's not very common on small projects.

The python guidance is:

• if you want keep all the library stuff under one directory -> Use packages with __init__ magic.
• If you want to expose a single package that uses code in multiple directories --> Namespace packages were created for that reason.

[gilles-peskine-arm]

If you want to expose a single package that uses code in multiple directories

We might want to do that: we haven't figured out yet. It depends how we organize scripts whose code is spread into up to three repositories (crypto, tls and framework).

If namespaces give us some flexibility, it doesn't matter how complicated they are under the hood. Are they complicated for people who will create new modules, create scripts that use modules, and run checkers (pylint/mypy)?

[minosgalanakis]

If namespaces give us some flexibility, it doesn't matter how complicated they are under the hood. Are they complicated for people who will create new modules, create scripts that use modules, and run checkers (pylint/mypy)?

That unfortunately I do not know. I never had to reach that level of complexity in a project I was working on. But I believe it shouldn't be an issue for people creating modules or scripts, as long as they are placed in the right location ( you set the folder structure and the contents are sorted accordingly )

For pylint/mympy I suspect there coud be issues 1 since those tools focus on testing simple use-cases. But because namespaces exist for a while we could be ok by updating the version we use. e.g 2

The best way to go about it is trying to create a mock namespace, and then see it works with our enviroment.