Development of dependent roles requires workarounds because of non configurable symlinking

[jimboolio]

Prerequisites

• This was not already reported in the past (duplicate check)
• It does reproduce it with code from main branch (latest unreleased version)
• I include a minimal example for reproducing the bug
• The bug is not trivial, as for those a direct pull-request is preferred
• Running pip check does not report any conflicts
• I was able to reproduce the issue on a different machine
• The issue is not specific to any driver other than 'default' one

Environment

molecule 6.0.2 using python 3.11 
    ansible:2.15.4
    azure:23.5.0 from molecule_plugins
    containers:23.5.0 from molecule_plugins requiring collections: ansible.posix>=1.3.0 community.docker>=1.9.1 containers.podman>=1.8.1
    default:6.0.2 from molecule
    docker:23.5.0 from molecule_plugins requiring collections: community.docker>=3.0.2 ansible.posix>=1.4.0
    ec2:23.5.0 from molecule_plugins
    gce:23.5.0 from molecule_plugins requiring collections: google.cloud>=1.0.2 community.crypto>=1.8.0
    podman:23.5.0 from molecule_plugins requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
    vagrant:23.5.0 from molecule_plugins

On Fedora 38, everything installed in python virtual environment

What happened

If developing for example role b with molecule which depends on another role a, both of which developed with molecule, first a then b

Summary of issues

1. the dependency install of role b will crash because the role a has symlink to ~/.ansible/roles and ansible galaxy cannot handle that, if it could, it would overwrite uncommitted changes in role a
2. symlinking to ~/.ansible/roles cannot be disabled
3. The issue 1 can be worked around with shell provisioner together with ANSIBLE_ROLES_PATH in molecule.yml, this works if requirements.yml file does not exist
4. when setting molecule dependency management to shell, it unexpectedly still uses ansible galaxy before calling the shell script if requirements.yml file exists in a role root
5. Command molecule reset will fail if using other than the docker provider (fails because molecule[docker] not installed)

Details

3.

I had an idea of using my development directory as ANSIBLE_ROLES_PATH, so in the examples the parent of ansible_a and parent of ansible_b when both of those roles live in the same development directory next to each other. I set the env variable in molecule.yml to equal "../../../" since it seems like the default value is the molecule scenario.

This approach will still generate the symlinks, meaning that if not careful, there's a risk of running development versions of role a and/or role b since it creates those symlinks in the background and there's only one line warning labeled as info.

And at some stage I even managed to run production versions of my roles against my test environment, meaning the tests passed with false positives

4.

If I switch the dependency handler from ansible galaxy to shell, molecule still attempts to install the requirements.yml dependencies which is unwanted, that's why I have replaced the dependency option from galaxy to shell

dependency:
  name: shell
  command: ./dependency.sh

If the above is in molecule.yml, but a requirements.yml file exists, ansible galaxy will be called automatically. It ignores the custom role path, tries to overwrite the symlinked role a and crashes.

5.

I included this here since I'm not sure if this is a proper issue

Steps and repos to reproduce

I was not able to get latest unreleased molecule to work in my virtual environment, sorry.

Reproducing 1.

Run molecule tests on role a: https://github.com/jimboolio/myorg.ansible_a

Then try running tests on this role b: https://github.com/jimboolio/myorg.ansible_b

Reproducing 5.

Continuing on the same environment as on issue 1. reproduction.

Remove the requirements.yml file from role b, molecule test won't crash now, but if using vagrant provider, molecule reset command will now fail in role b since molecule[docker] is not installed:

.../venv/lib64/python3.11/site-packages/molecule_plugins/docker/driver.py", line 256, in reset
    import docker
ModuleNotFoundError: No module named 'docker'

Reproducing 4.

https://github.com/jimboolio/myorg.ansible_requirements

The issue is apparent when the requirement.yml syntax is valid but the url points to some nonexistent role source url. Nothing is configured to use the requirements.yml in the shell dependency management script but molecule test still fails since the role isn't installable

Reproducing example

No response