Using the GitLab Shell chart

The gitlab-shell sub-chart provides an SSH server configured for Git SSH access to GitLab.

Requirements

This chart depends on access to the Workhorse services, either as part of the complete GitLab chart or provided as an external service reachable from the Kubernetes cluster this chart is deployed onto.

Design Choices

In order to easily support SSH replicas, and avoid using shared storage for the SSH authorized keys, we are using the SSH AuthorizedKeysCommand to authenticate against the GitLab authorized keys endpoint. As a result, we don’t persist or update the AuthorizedKeys file within these pods.

Configuration

The gitlab-shell chart is configured in two parts: external services, and chart settings. The port exposed through Ingress is configured with global.shell.port, and defaults to 22. The Service’s external port is also controlled by global.shell.port.

Installation command line options

Chart configuration examples

extraEnv

extraEnv allows you to expose additional environment variables in all containers in the pods.

Below is an example use of extraEnv:

extraEnv:
  SOME_KEY: some_value
  SOME_OTHER_KEY: some_other_value

When the container is started, you can confirm that the environment variables are exposed:

env | grep SOME
SOME_KEY=some_value
SOME_OTHER_KEY=some_other_value

extraEnvFrom

extraEnvFrom allows you to expose additional environment variables from other data sources in all containers in the pods.

Below is an example use of extraEnvFrom:

extraEnvFrom:
  MY_NODE_NAME:
    fieldRef:
      fieldPath: spec.nodeName
  MY_CPU_REQUEST:
    resourceFieldRef:
      containerName: test-container
      resource: requests.cpu
  SECRET_THING:
    secretKeyRef:
      name: special-secret
      key: special_token
      # optional: boolean
  CONFIG_STRING:
    configMapKeyRef:
      name: useful-config
      key: some-string
      # optional: boolean

image.pullSecrets

pullSecrets allows you to authenticate to a private registry to pull images for a pod.

Additional details about private registries and their authentication methods can be found in the Kubernetes documentation.

Below is an example use of pullSecrets:

image:
  repository: my.shell.repository
  tag: latest
  pullPolicy: Always
  pullSecrets:
  - name: my-secret-name
  - name: my-secondary-secret-name

serviceAccount

This section controls if a ServiceAccount should be created and if the default access token should be mounted in pods.

livenessProbe/readinessProbe

deployment.livenessProbe and deployment.readinessProbe provide a mechanism to help control the termination of Pods under some scenarios.

Larger repositories benefit from tuning liveness and readiness probe times to match their typical long-running connections. Set readiness probe duration shorter than liveness probe duration to minimize potential interruptions during clone and push operations. Increase terminationGracePeriodSeconds and give these operations more time before the scheduler terminates the pod. Consider the example below as a starting point to tune GitLab Shell pods for increased stability and efficiency with larger repository workloads.

deployment:
  livenessProbe:
    initialDelaySeconds: 10
    periodSeconds: 20
    timeoutSeconds: 3
    successThreshold: 1
    failureThreshold: 10
  readinessProbe:
    initialDelaySeconds: 10
    periodSeconds: 5
    timeoutSeconds: 2
    successThreshold: 1
    failureThreshold: 3
  terminationGracePeriodSeconds: 300

Reference the official Kubernetes Documentation for additional details regarding this configuration.

tolerations

tolerations allow you schedule pods on tainted worker nodes

Below is an example use of tolerations:

tolerations:
- key: "node_label"
  operator: "Equal"
  value: "true"
  effect: "NoSchedule"
- key: "node_label"
  operator: "Equal"
  value: "true"
  effect: "NoExecute"

affinity

For more information, see affinity.

annotations

annotations allows you to add annotations to the GitLab Shell pods.

Below is an example use of annotations

annotations:
  kubernetes.io/example-annotation: annotation-value

External Services

This chart should be attached the Workhorse service.

Workhorse

workhorse:
  host: workhorse.example.com
  serviceName: webservice
  port: 8181

Chart settings

The following values are used to configure the GitLab Shell Pods.

hostKeys.secret

The name of the Kubernetes secret to grab the SSH host keys from. The keys in the secret must start with the key names ssh_host_ in order to be used by GitLab Shell.

authToken

GitLab Shell uses an Auth Token in its communication with Workhorse. Share the token with GitLab Shell and Workhorse using a shared Secret.

authToken:
 secret: gitlab-shell-secret
 key: secret

LoadBalancer Service

If the service.type is set to LoadBalancer, you can optionally specify service.loadBalancerIP to create the LoadBalancer with a user-specified IP (if your cloud provider supports it).

You can also optionally specify a list of service.loadBalancerSourceRanges to restrict the CIDR ranges that can access the LoadBalancer (if your cloud provider supports it).

Additional information about the LoadBalancer service type can be found in the Kubernetes documentation

service:
  type: LoadBalancer
  loadBalancerIP: 1.2.3.4
  loadBalancerSourceRanges:
  - 5.6.7.8/32
  - 10.0.0.0/8

OpenSSH supplemental configuration

When making use of OpenSSH’s sshd (via .sshDaemon: openssh), it is possible to provide supplemental configuration in two ways: .opensshd.supplemental_config, and via mounting configuration snippets to /etc/ssh/sshd_config.d/*.conf.

Any configuration supplied must meet the functional requirements of sshd_config. Ensure you read the manual page.

opensshd.supplemental_config

The content of .opensshd.supplemental_config will be directly placed at the end the sshd_config file within the container. This value should be a mutli-line string.

Example, enabling older clients using the ssh-rsa key exchange algorithms. Note that enabling deprecated algorithms, such as ssh-rsa, creates significant security vulnerabilities. The likelihood of exploitation is significantly amplified on publicly exposed GitLab instances with these changes.

opensshd:
    supplemental_config: |-
      HostKeyAlgorithms +ssh-rsa,ssh-rsa-cert-v01@openssh.com
      PubkeyAcceptedAlgorithms +ssh-rsa,ssh-rsa-cert-v01@openssh.com
      CASignatureAlgorithms +ssh-rsa

sshd_config.d

You may provide full configuration snippets to sshd via mounting content into /etc/ssh/sshd_config.d, with the files matching *.conf. Note, that these are included after the default configuration which is required for the application to function in the container, and within the chart. These values will not override the contents of sshd_config, but extend them.

Example, mounting a single item of a ConfigMap into the container via extraVolumes and extraVolumeMounts:

extraVolumes: |
  - name: gitlab-sshdconfig-extra
    configMap:
      name: gitlab-sshdconfig-extra

extraVolumeMounts: |
  - name: gitlab-sshdconfig-extra
    mountPath: /etc/ssh/sshd_config.d/extra.conf
    subPath: extra.conf

Configuring the networkpolicy

This section controls the NetworkPolicy. This configuration is optional and is used to limit Egress and Ingress of the Pods to specific endpoints.

Example Network Policy

The gitlab-shell service requires Ingress connections for port 22 and Egress connections to various to default workhorse port 8181. This example adds the following network policy:

• Allows Ingress requests:

  • From the nginx-ingress pod to port 2222

  • From the prometheus pod to port 9122

    Access from prometheus to port 9122 is only necessary when the SSH daemon is set to gitlab-sshd

• Allows Egress requests:

  • To the webservice pod to port 8181
  • To the gitaly pod to port 8075

Note the example provided is only an example and may not be complete

The example is based on the assumption that kube-dns was deployed to the namespace kube-system, prometheus was deployed to the namespace monitoring and nginx-ingress was deployed to the namespace nginx-ingress.

networkpolicy:
  enabled: true
  ingress:
    enabled: true
    rules:
      - from:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: nginx-ingress
            podSelector:
              matchLabels:
                app: nginx-ingress
                component: controller
        ports:
          - port: 2222
      - from:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: monitoring
            podSelector:
              matchLabels:
                app: prometheus
                component: server
                release: gitlab
        ports:
          - port: 9122
  egress:
    enabled: true
    rules:
      - to:
          - podSelector:
              matchLabels:
                app: gitaly
        ports:
          - port: 8075
      - to:
          - podSelector:
              matchLabels:
                app: webservice
        ports:
          - port: 8181
      - to:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: kube-system
            podSelector:
              matchLabels:
                k8s-app: kube-dns
        ports:
          - port: 53
            protocol: UDP

Configuring KEDA

This keda section enables the installation of KEDA ScaledObjects instead of regular HorizontalPodAutoscalers. This configuration is optional and can be used when there is a need for autoscaling based on custom or external metrics.

Most settings default to the values set in the hpa section where applicable.

If the following are true, CPU and memory triggers are added automatically based on the CPU and memory thresholds set in the hpa section:

• triggers is not set.
• The corresponding request.cpu.request or request.memory.request setting is also set to a non-zero value.

If no triggers are set, the ScaledObject is not created.

Refer to the KEDA documentation for more details about those settings.

See examples/keda/gitlab-shell.yml for an usage example of keda.