Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot use Deployer with submodules #407

Open
6 tasks done
FelipeEmerim opened this issue Sep 6, 2019 · 9 comments
Open
6 tasks done

Cannot use Deployer with submodules #407

FelipeEmerim opened this issue Sep 6, 2019 · 9 comments

Comments

@FelipeEmerim
Copy link

FelipeEmerim commented Sep 6, 2019 •
edited

Before submitting your issue, please make sure that you've checked all of the checkboxes below.

  • You're running the latest release version of Deployer.
  • Ensure that you're running at least PHP 7.0.8, you can check this by running php -v
  • You've ran composer install --no-dev from the root of your installation.
  • You've ran npm install --production from the root of your installation.
  • You've ran rm -rf bootstrap/cache/* from the root of your installation.
  • You have restarted the queue listener and node socket server.

Describe the bug

When trying to use deployer with a repository that has submodules. The deploy fails with the following error: permission denied (publickey) while cloning the submodule.

Expected behaviour

It would properly deploy the project.

Actual behaviour

The deploy fails due to the error mentioned above without event entering the first step.

Steps to reproduce

  • Include a project with submodule in deployer.
  • Configure the same ssh key in the master and submodule projects.
  • Try to deploy a version of the project.

Environment info

Laravel Environment

  • Laravel Version: 5.5.41
  • Timezone: America/Sao_Paulo
  • Debug Mode: ✔
  • Storage Dir Writable: ✔
  • Cache Dir Writable: ✔
  • Decomposer Version: ^1.0
  • App Size: 2 GB

Server Environment

  • PHP Version: 7.0.33-0+deb9u3
  • Server Software: nginx/1.10.3
  • Server OS: Linux magneto 3.16.0-6-amd64 Route models #1 SMP Debian 3.16.57-2 (2018-07-14) x86_64
  • Database: pgsql
  • SSL Installed: ✘
  • Cache Driver: file
  • Session Driver: file
  • Openssl Ext: ✔
  • PDO Ext: ✔
  • Mbstring Ext: ✔
  • Tokenizer Ext: ✔
  • XML Ext: ✘
  • Curl Ext: ✘
  • GD Ext: ✔
  • JSON Ext: ✔

Installed Packages & their version numbers

  • andywer/js-localization : dev-laravel-5
  • backup-manager/laravel : ^1.1
  • creativeorange/gravatar : ~1.0
  • doctrine/dbal : 2.5.13
  • fideloper/proxy : ^3.3
  • graham-campbell/binput : ~4.0
  • guzzlehttp/guzzle : ^6.2
  • htmlmin/htmlmin : ~5.0
  • intervention/image : ^2.3
  • laracademy/interactive-make : ^1.1
  • laravel-notification-channels/hipchat : ~1.0
  • laravel-notification-channels/twilio : ~2.0.1
  • laravel-notification-channels/webhook : ~1.1.0
  • laravel/framework : ^5.5
  • laravel/tinker : 1.0.*
  • lubusin/laravel-decomposer : ^1.0
  • mccool/laravel-auto-presenter : ^6.2
  • melihovv/laravel-log-viewer : ^5.5
  • michele-angioni/multi-language : 0.4
  • pda/pheanstalk : ~3.1
  • pragmarx/google2fa : ^1.0.0
  • predis/predis : ~1.0
  • rebelinblue/laravel5-zxcvbn : ^1.0.0
  • tymon/jwt-auth : ~0.5
  • version/version : ^2.2

Extra Information

  • proc_open enabled : ✔

Logs (see storage/logs/) or other output that would be helpful

[2019-09-05 20:26:38] development.DEBUG: ### Run script locally
bash -s << 'EOF'
    # Turn on quit on non-zero exit
    set -e
    ### Mirror git repository
chmod +x /var/www/deployer/storage/app/tmp/ssh5W8wC8
export GIT_SSH=/var/www/deployer/storage/app/tmp/ssh5W8wC8

if [ ! -d /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git ]; then
    git clone --mirror git@bitbucket.org:VE3/colplan.git /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git
fi

cd /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git

git fetch --all --prune
EOF
  
[2019-09-05 20:26:41] development.DEBUG: ### Run script locally
bash -s << 'EOF'
    # Turn on quit on non-zero exit
    set -e
    ### Get commit details - 1365
cd /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git

git log master -n1 --pretty=format:"%H%x09%an%x09%ae"
EOF
  
[2019-09-05 20:26:41] development.DEBUG: ### Run script locally
bash -s << 'EOF'
    # Turn on quit on non-zero exit
    set -e
    ### Create release archive - 1365
git clone --depth 1 --recursive /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git /var/www/deployer/storage/app/tmp/clone_17_20190905202638
cd /var/www/deployer/storage/app/tmp/clone_17_20190905202638
git checkout 36cc87b2cbb01f96473c80abce76c85081a871f0
/var/www/deployer/resources/scripts/tools/GitArchiveAll.sh --tree-ish 36cc87b2cbb01f96473c80abce76c85081a871f0 --format tar.gz --verbose /var/www/deployer/storage/app/17_20190905202636.tar.gz
cd -
rm -rf /var/www/deployer/storage/app/tmp/clone_17_20190905202638
EOF
  
[2019-09-05 20:26:41] development.DEBUG: ### Run script locally
bash -s << 'EOF'
    # Turn on quit on non-zero exit
    set -e
    ### List git references
cd /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git

git tag --list --no-column
EOF
  
[2019-09-05 20:26:43] development.DEBUG: ### Run script locally
bash -s << 'EOF'
    # Turn on quit on non-zero exit
    set -e
    ### List git references
cd /var/www/deployer/storage/app/mirrors/git_bitbucket.org_VE3_colplan.git

git branch --list --no-column
EOF
  
[2019-09-05 20:26:45] development.ERROR: Cloning into '/var/www/deployer/storage/app/tmp/clone_17_20190905202638'...
warning: --depth is ignored in local clones; use file:// instead.
done.
Submodule 'forecaster' (git@bitbucket.org:VE3/forecaster.git) registered for path 'forecaster'
Cloning into '/var/www/deployer/storage/app/tmp/clone_17_20190905202638/forecaster'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@bitbucket.org:VE3/forecaster.git' into submodule path '/var/www/deployer/storage/app/tmp/clone_17_20190905202638/forecaster' failed
Failed to clone 'forecaster'. Retry scheduled
Cloning into '/var/www/deployer/storage/app/tmp/clone_17_20190905202638/forecaster'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@bitbucket.org:VE3/forecaster.git' into submodule path '/var/www/deployer/storage/app/tmp/clone_17_20190905202638/forecaster' failed
Failed to clone 'forecaster' a second time, aborting

When nailing down the issue I found that cloning a submodule from a mirror requires authentication. So the ssh configs script should be passed as an argument to the CreateReleaseArchive.sh which should export GIT_SSH as you do when you clone the mirror.

If you do not need a submodule just remove --recursive from the git clone command.
@REBELinBLUE
Copy link
Owner

REBELinBLUE commented Sep 6, 2019 via email

@FelipeEmerim
Copy link
Author

Yes, same private key on the two repos. I can clone them as long as I pass the ssh key. It seems that the deployer does not use the wrapper ssh script when cloning from the mirror which raises the error.

@FelipeEmerim
Copy link
Author

FelipeEmerim commented Sep 6, 2019 •
edited

A simple test:

#!/usr/bin/env sh

ssh -o CheckHostIP=no \
    -o IdentitiesOnly=yes \
    -o StrictHostKeyChecking=no \
    -o PasswordAuthentication=no \
    -o IdentityFile=MyprivateKey $*`

export GIT_SSH=/path/to/script.sh

git clone --mirror git@bitbucket.org:VE3/repoWithSubmodules.git

This is what deployer does to clone the mirror and it is working. Now when it clones from the mirror to the repository to archive, it does not set the git_ssh var so:

unset GIT_SSH

git clone --depth 1 --recursive projectWithSubmdules.git projectWithSubmodules

This last command will raise the mentioned error.

@vhlavsa
Copy link

vhlavsa commented Mar 29, 2020

Hi everybody, I have a same problem. Project with submodules, all of them have access keys same as the main project, but deploy keeps failing.

Is there any way to make it work without getting rid of the submodules?

Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.
fatal: clone of 'git@bitbucket.org:xyz.git' into submodule path 'xyz' failed

@FelipeEmerim
Copy link
Author

I tried some workarounds but none of them worked. In our case, since the submodule is actually a SOAP service we deployed it separately.

@darmen
Copy link

darmen commented Jun 16, 2022

Hey there 👋🏼

I've encountered similar problem some time ago, and ended up writing a quick tip on how I approached it – https://heydarmen.com/blog/using-deployer-with-git-submodules.html

@REBELinBLUE
Copy link
Owner

hey @darmen I think you are looking for https://github.com/deployphp/deployer

@darmen
Copy link

darmen commented Jun 17, 2022

Sorry @REBELinBLUE.

@REBELinBLUE
Copy link
Owner

No worries :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@darmen @REBELinBLUE @vhlavsa @FelipeEmerim