How to Scale out Gitlab EE

Question

Currently I am running the whole gitlab EE as a single container. I need to scale out the service so that it can support more users and more operations/pull/push/Merge Requests etc simultanously.

• I need to run a redis cluster of its own

• I need to run a PG cluster separate

• I need to integrate elasticsearch for search

But how can I scale out the remaning core gitlab services. Do they support a scale out architecture.

• gitlab workhorse
• unicorn ( gitlab rails )
• sidekiq ( gitlab rails )
• gitaly
• gitlab shell

Answer 1

Do they support a scale out architecture.

Not exactly, considering the GitLab Omnibus image is one package with bundled dependencies.

But I never experienced so much traffic that it needed to be split up and scaled out.

There is though a proposal for splitting up the Omnibus image: gitlab-org/omnibus-gitlab issue 1800.

It points out to gitlab-org/build/CNG which does just what you are looking for:

Each directory contains the Dockerfile for a specific component of the infrastructure needed to run GitLab.

• rails - The Rails code needed for both API and web.
• unicorn - The Unicorn container that exposes Rails.
• sidekiq - The Sidekiq container that runs async Rails jobs
• shell - Running GitLab Shell and OpenSSH to provide git over ssh, and authorized keys support from the database
• gitaly - The Gitaly container that provides a distributed git repos

The other option, using Kubernetes, is the charts/gitlab:

The gitlab chart is the best way to operate GitLab on Kubernetes. This chart contains all the required components to get started, and can scale to large deployments.
Some of the key benefits of this chart and corresponding containers are:

• Improved scalability and reliability
• No requirement for root privileges
• Utilization of object storage instead of NFS for storage

The default deployment includes:

• Core GitLab components: Unicorn, Shell, Workhorse, Registry, Sidekiq, and Gitaly
• Optional dependencies: Postgres, Redis, Minio
• An auto-scaling, unprivileged GitLab Runner using the Kubernetes executor
• Automatically provisioned SSL via Let's Encrypt.

---

Update Sept. 2020:

GitLab 13.4 offers one feature which can help scaling out GitLab on-premise:

Gitaly Cluster majority-wins reference transactions (beta)

Gitaly Cluster allows Git repositories to be replicated on multiple warm Gitaly nodes. This improves fault tolerance by removing single points of failure.

Reference transactions, introduced in GitLab 13.3, causes changes to be broadcast to all the Gitaly nodes in the cluster, but only the Gitaly nodes that vote in agreement with the primary node persist the changes to disk.
If all the replica nodes dissented, only one copy of the change would be persisted to disk, creating a single point of failure until asynchronous replication completed.

Majority-wins voting improves fault tolerance by requiring a majority of nodes to agree before persisting changes to disk. When the feature flag is enabled, writes must succeed on multiple nodes. Dissenting nodes are automatically brought in sync by asynchronous replication from the nodes that formed the quorum.

See Documentation and Issue.