Dirk.Net

If you get errors such as

fatal: Out of memory, malloc failed
error: pack-objects died with strange error
error: failed to push some refs to ‘git@myserver.com:foo.git’

when running

git push

first, upgrade git to the latest version on both ends, then check if

git repack

still works. If not, you might have a local issue.

If repack still works, your remote server (the server you push to) might actually run out of memory, which can easily happen with for example smallish VPS slices. So, just increase your swap file as Linus suggests, like this and it might start working again. Hope it helps.

Update: Two hours after this post I discovered that the system clock on the remote server, just recently booked from my reliable hosting company, was off by about eight hours. On a server set up by them just a few days ago. Anyway, the clock issue might be the actual underlying reason for the failed authentication.  So check your clock and sync it. I guess the name of the lone horseman is Crystaltech. Duh.

Original post:

Ok, the title may be a bit harsh, but that’s how I felt at times over the last two days: Some evil forces slicing days out of my life without me getting much in return – besides learning a lot. 

So if you follow the instructions here and here  with perhaps a little sprinkling or CentOS you set up the most powerful web app deployment solution available today, a combination of your own git repository server (gitosis) and powerful deployment scripts (Capistrano). Then, after gitosis administration via gitosis-admin.git works like magic and cap deploy:setup runs without issues, you run cap deploy:cold.

And you need old git’s password?!! The gitosis user you’ve painstakingly set up without as password, i.e. the —disabled-password option (-r in CentOS), so who is positively devoid of a password, and you still get

 ** [myhost.com :: out]
 ** [myhost.com :: out] Permission denied, please try again.
 ** [myhost.com :: out] git@myhost.com’s password:
Password:
 ** [myhost.com :: out]
 ** [myhost.com :: out] Permission denied (publickey,gssapi-with-mic,password).
 ** [myhost.com :: out] fatal: The remote end hung up unexpectedly
    command finished

ad nauseam. And in your log (/var/log/secure for CentOS) you’ll have

Mar  1 04:27:25 HOST1 sshd[12653]: Postponed publickey for git from 210.11.11
.11 port 25409 ssh2
Mar  1 04:27:26 HOST1 sshd[12652]: Accepted publickey for git from 210.11.11.
11 port 25409 ssh2
Mar  1 04:27:30 HOST1 sshd[12657]: Accepted publickey for deploy from 210.11.1
1.11 port 25412 ssh2
Mar  1 04:28:15 HOST1 sshd[12663]: Failed password for git from 63.22.22.22
 port 34759 ssh2
Mar  1 04:28:16 HOST1 sshd[12663]: Failed password for git from 63.22.22.22
 port 34759 ssh2

where the 210.11.11.11 is the ip of your development machine, and 63.22.22.22 is the ip of your server running gitosis which is also the deployment target server (This ip-difference made me think of iptables, oh well.).

Now, before you mess with

• iptables
• ssh-agent
• sshd_config
• authorized_keys and .ssh permissions
• reinstalling and upgrading gitosis and other stuff

and anything else you might try, first, simply add

on :start do
    `ssh-add`
end

(mind the backquotes) to your deploy.rb file. And it might just start working. Here’s the solution, with the original suggestion here. I re-discovered it as I was calling it a night, closing one of the 132 Firefox tabs I had opened in this effort.

The issue is probably not Capistrano’s fault but has to do with the specific ssh setup on the server. I had no such problems with an Ubuntu slice as now on a dedicated CentOS, perhaps due to a more severe security lockdown on this “Enterprise” distro.  However, it didn’t make any difference for me if ssh-agent was running or if I had

default_run_options[:pty] = true

in my deploy.rb file or or or…. YMMV.

First and foremost, read this document and follow it meticulously. If you get this error:

unable to chdir or not a git archive
fatal: The remote end hung up unexpectedly

after running

cap deploy:cold

you might have to add the public key of the deploy user you set up with

set :user, “deploy”

in your capistrano deploy.rb file, to the /home/git/.ssh/authorized_keys on the REMOTE server.

Of course gitosis has a special process managing the keys and users from the LOCAL machine in the ~/gitosis/gitosis-admin directory, so you to have copy the public key you generated for the deploy user on your REMOTE server via

ssh-keygen -t rsa

to your LOCAL machine (via scp or cut and paste into a new deploy.pub file from the ssh shell for example) and then proceed as outlined here (under “Adding Users”):

cd ~/gitosis/gitosis-admin
cp ~/deploy.pub keydir/
git add keydir/deploy.pub

Add the user credentials to the gitosis.conf file:

[group myrailsteam]
writable = myrailsapp
members = deploy

Finally run

git commit -a -m “Granted deploy user access rights to myrailsapp”
git push

Now the cap deploy:cold should work (with one fewer errors that is ).