Gitlab 7.5.3 not syncing SSH authorized_keys

Question

I upgraded from Gitlab 7.0 to 7.5.3 following the guidelines.

All went well but one issue remaining is that the automatic sync of sidekiq with gitlab-shell authorized_keys file is not working anymore. Neither add nor delete.

Log files did not state anything, after e.g. adding a key I only see GitlabShellWorker executing successfully: sidekiq.log:

2015-01-06T08:31:26Z 3292 TID-kjwek GitlabShellWorker JID-3b65f0ad00be156f30a22a62 INFO: start
2015-01-06T08:31:27Z 3292 TID-kjwek GitlabShellWorker JID-3b65f0ad00be156f30a22a62 INFO: done: 0.589 sec
2015-01-06T08:35:00Z 3292 TID-kjwek Sidekiq::Extensions::DelayedMailer JID-9c4837cd0acb34cb78cf5148 INFO: start
2015-01-06T08:35:28Z 3292 TID-kjwek Sidekiq::Extensions::DelayedMailer JID-9c4837cd0acb34cb78cf5148 INFO: done: 27.98 sec

No errors or hints in other logs.

System Information:

git@aux-git:~/gitlab> sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.9.1
Checking Environment ...

Git configured for git user? ... yes

Checking Environment ... Finished

Checking GitLab Shell ...

GitLab Shell version >= 2.2.0 ? ... OK (2.2.0)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
Satellites access is drwxr-x---? ... yes
hooks directories in repos are links: ...
2/30 ... ok
2/47 ... ok
5/51 ... ok
2/53 ... ok
5/55 ... ok
2/56 ... ok
5/57 ... ok
2/58 ... ok
10/60 ... ok
32/61 ... ok
6/62 ... ok
9/64 ... ok
6/65 ... ok
2/66 ... ok
2/67 ... ok
2/68 ... ok
2/69 ... ok
2/70 ... ok
41/72 ... ok
2/74 ... ok
2/76 ... ok
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files:
        /var/git/repositories: OK
        /home/git/.ssh/authorized_keys: OK
Test redis-cli executable: redis-cli 2.8.6
Send ping to redis server: PONG
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking LDAP ...

LDAP users with access to your GitLab server (only showing the first 100 results)
Server: ldapmain
[..]
Checking LDAP ... Finished

Checking GitLab ...

Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ...
2/30 ... yes
2/47 ... yes
5/51 ... yes
2/53 ... yes
5/55 ... yes
2/56 ... yes
5/57 ... yes
2/58 ... yes
10/60 ... yes
32/61 ... yes
6/62 ... yes
9/64 ... yes
6/65 ... yes
2/66 ... yes
2/67 ... yes
2/68 ... yes
2/69 ... yes
2/70 ... yes
41/72 ... yes
2/74 ... yes
2/76 ... yes
Projects have satellites? ...
2/30 ... yes
2/47 ... yes
5/51 ... yes
2/53 ... yes
5/55 ... yes
2/56 ... yes
5/57 ... yes
2/58 ... yes
10/60 ... yes
32/61 ... yes
6/62 ... yes
9/64 ... yes
6/65 ... yes
2/66 ... yes
2/67 ... yes
2/68 ... yes
2/69 ... yes
2/70 ... yes
41/72 ... yes
2/74 ... yes
2/76 ... yes
Redis version >= 2.0.0? ... yes
Ruby version >= 2.0.0 ? ... yes (2.0.0)
Your git bin path is "/usr/bin/git"
Git version >= 1.7.10 ? ... yes (1.8.4)

Checking GitLab ... Finished

git@aux-git:~/gitlab> sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.9.1

System information
System:         openSUSE project 13.1
Current User:   git
Using RVM:      no
Ruby Version:   2.0.0p247
Gem Version:    2.0.3
Bundler Version:1.6.2
Rake Version:   10.3.2
Sidekiq Version:2.17.0

GitLab information
Version:        7.5.3
Revision:       b656b85
Directory:      /home/git/gitlab
DB Adapter:     mysql2
URL:            http://aux-git
HTTP Clone URL: http://aux-git/some-project.git
SSH Clone URL:  git@aux-git:some-project.git
Using LDAP:     yes
Using Omniauth: no

GitLab Shell
Version:        2.2.0
Repositories:   /var/git/repositories/
Hooks:          /home/git/gitlab-shell/hooks/
Git:            /usr/bin/git

Any ideas? Is it a Gitlab bug? I saw some changes in shell.rb since 7.0.

Thanks Christian

Answer 1

So, sidekiq got updated from 2.17.1 to 2.17.2, and the changelog mentions a bug fix that changes how sidekiq can be invoked.

It's likely that your GitLab installation is still invoking sidekiq using the old style, something like the following.

sidekiq -q post_receive,mailer,system_hook,project_web_hook,gitlab_shell,common,default -e production

This no longer works. If that's the case you need to fix your launch scripts to invoke sidekiq as follows, instead.

sidekiq -q post_receive -q mailer -q system_hook -q project_web_hook -q gitlab_shell -q common -q default -e production